Web Scraping
A web scraping test site with known, predictable markup. Practice pages for crawlers, scrapers, and browser automation, from a full ecommerce catalog to pagination, forms, and login walls.
Ecommerce catalog
A fake ecommerce store to scrape: category listings down to individual product detail pages, the classic web-scraping example.
- easy Store home — Browse every product category in the fake store.
- easy Category listing — Products within one category, each linking to its own detail page.
- easy Product detail page — A realistic ecommerce scraping target, with price, rating, stock, and related items.
Structured data & metadata
Machine-readable product, collection, and business data in the formats real sites use: JSON-LD, Microdata, Open Graph, and JS dataLayers.
- easy Format overview — The same demo product in every format below, for comparing extraction results.
- easy Product — JSON-LD — Product data exposed only via a schema.org/Product JSON-LD script.
- easy Product — Microdata — Same product, marked up with schema.org Microdata (itemscope/itemprop) instead of JSON-LD.
- easy Product — Open Graph — Same product, exposed only via Open Graph / product: meta tags in <head>.
- medium Product — dataLayer — Same product pushed into window.dataLayer as a GA4/GTM ecommerce event. No markup, so JS execution is required.
- medium Product — combined — JSON-LD, Microdata, Open Graph, and dataLayer all present at once (a realistic messy page).
- easy Collection (ItemList) — A category page exposed as a schema.org CollectionPage/ItemList.
- easy Product reviews — A product with visible reviews plus Review/AggregateRating JSON-LD.
- easy FAQ page — Common questions marked up as a schema.org FAQPage.
- easy Contact page — Email, phone, and address as plain text, mailto:/tel: links, Microdata, and Organization JSON-LD.
Pagination
Different pagination patterns for practicing crawling multi-page listings.
- easy Numbered pagination — Classic ?page=N style listing, 10 items per page, with rel="next"/"prev" link tags in <head>.
- easy Offset/limit pagination — Query-param offset/limit pagination.
- medium Load more button — Fetches additional items via a JSON API on button click.
- medium Infinite scroll — Loads more items automatically as you scroll.
Rendering modes
Content delivered differently: server-rendered vs. client-rendered.
- hard JS-rendered content — Empty shell on load; content is injected via a client-side fetch. Requires a JS-capable scraper.
- hard Frames & iFrames — Content nested inside an iframe pointing at a separate URL, two levels deep. Find the real page and fetch it directly.
Tables
Tabular data in a few different shapes.
- easy Simple table — A single flat product table.
- medium Sortable table — Client-navigable sortable columns via ?sort=&dir= query params.
- hard Nested/merged cells — Table grouped by category using rowspan.
Forms
Forms with varying complexity and hidden fields.
- easy Basic GET form — Simple search form, submits via GET.
- easy POST form — Form submission via POST with a confirmation page.
- hard Multi-step form — Three-step wizard using a session cookie to persist state.
- medium File upload — Upload form with a file input.
- hard Hidden & honeypot fields — Form with a CSRF token and a bot-trap honeypot field that must stay empty.
Authentication
Cookie/session-based login wall.
- medium Login page — Sets a session cookie on successful login (user: demo / pass: demo).
- medium Protected dashboard — Redirects to /login unless a valid session cookie is present.
- easy Logout — Clears the session cookie.
Dynamic elements
Elements that change between requests or after a delay, useful for checking selector robustness and explicit-wait logic.
- medium Dynamic elements — A randomized id on every request, plus buttons that only become enabled or change color after a delay.
Accessibility
The same page with and without intentional accessibility issues, documented on-page. Built for automated a11y scanners like axe-core, pa11y, or Lighthouse.
- medium Inaccessible version — Four documented, scanner-detectable issues: missing alt text, an unlabeled input, low contrast, and a skipped heading level.
- easy Accessible version — The same content with every issue above fixed, for comparison.
Link graphs
Crawling practice: pages for testing crawlers and link-following logic, including redirect chains, loops, and intentional dead ends.
- medium Redirect chain — Follows N sequential redirects before landing. Try /redirect-chain/{n}.
- medium Circular links — Three pages that link back to each other in a loop.
- easy Broken links — A page mixing valid links with intentional 404s.
Markup difficulty levels
The same product data rendered with progressively messier HTML.
- easy Clean semantic HTML — Well-structured HTML with meaningful classes.
- medium Div soup — Same content, all divs/spans, no semantic tags or classes.
- hard Obfuscated classes — Same content, hashed/randomized-looking class names.
- medium Data-attribute hooks — Same content, data extracted via data-* attributes rather than classes.
URL structure & normalization
The mechanics behind duplicate-content URLs: trailing slashes, case sensitivity, query parameter order, tracking params, and the <base> tag.
- easy Overview — Links to every URL-structure scenario below.
- easy Trailing slash — The same content at two URLs, one with a trailing slash, one canonical to the other.
- medium Case sensitivity — Paths are case-sensitive here; the lowercase variant of this URL is a genuine 404.
- medium Query parameter order & tracking params — A canonical tag computed from the request, sorting params and stripping known tracking params.
- medium Encoded characters — Shows the raw vs. decoded query string: %20, +, and a literal space.
- hard The <base> tag — A <base href> one directory above the page's own URL, so a relative link resolves somewhere unexpected.
Canonical tags & meta robots directives
Canonical tag scenarios (self, duplicate, conflicting, cross-domain) and page-level robots directives via both a meta tag and the X-Robots-Tag header.
- easy Overview — Links to every canonical and meta robots scenario below.
- easy Canonical: self-referencing — The recommended default: a page canonical to itself.
- medium Canonical: duplicate content, page A — The preferred URL in a duplicate-content pair.
- medium Canonical: duplicate content, page B — A different URL with the same content, canonical to page A.
- hard Canonical: tag vs. header conflict — The <link> tag names one URL, the HTTP Link header names another.
- medium Canonical: cross-domain — Canonical points at a URL on a different domain, the syndicated-content pattern.
- easy Meta robots: noindex, follow — Keep this page out of the index, but still crawl its links.
- easy Meta robots: index, nofollow — Index this page, but don't follow its outbound links.
- medium X-Robots-Tag header — The same noindex directive as an HTTP header instead of a meta tag, the only option for non-HTML resources.
- easy Meta robots: noarchive, nosnippet — Index this page, but don't cache a visible copy or show a snippet.
hreflang
Three language variants of the same page with reciprocal hreflang tags to each other, themselves, and an x-default.
- easy Overview — Links to all three language variants.
- medium English variant — hreflang="en", linking to the Spanish and French variants plus x-default.
Redirect mechanisms
Redirects that don't use a real HTTP 3xx status code, plus a genuine infinite redirect loop for testing loop-detection logic.
- easy Overview — Links to every redirect-mechanism scenario below.
- medium Meta refresh — A <meta http-equiv="refresh"> tag; the server response itself is a plain 200.
- hard JavaScript redirect — window.location set from a script, invisible to a scraper that skips JS execution.
- hard Infinite redirect loop — Two real HTTP 302s that point at each other forever, on purpose.