« Web Scraping

Redirect mechanisms

/http/redirect covers real HTTP 3xx status codes, the mechanism every HTTP client follows automatically. These pages cover the other ways a page can send a visitor elsewhere, each requiring different handling from a scraper.

• Meta refresh — a <meta http-equiv="refresh"> tag; the server always returns 200, only a refresh-aware client moves on
• JavaScript redirect — window.location set from a script; invisible to any scraper that doesn't execute JS
• Infinite redirect loop — two real HTTP 302s that point at each other forever, on purpose