Breadcrumb Trail Scraper

Universal Web Scraper & Data Extractor

motivational_nickel/universal-scraper

Fast AI-ready webpage extraction with structured JSON output for automation workflows, AI agents, RAG pipelines, SEO analysis, and website intelligence.

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Build an internal-link graph and identify sitemap orphan candidates.

Extract HTML table rows with multi-level headers and bounded rowspan/colspan expansion.

Extract public business locations, postal addresses, coordinates and location-page URLs.

Bulk URL Status & Redirect Checker (Free): Broken Links, SEO

euroscrape/bulk-url-checker-free

Free bulk URL checker: HTTP status code, full redirect chain (301, 302, 307, 308), final URL, response time and SEO tags (title, meta description, canonical, robots, H1) for up to 100,000 URLs. Find broken links, 404s, redirect loops and noindex pages after a migration. Export CSV, JSON.

Extract public product and category pages into a normalized product feed.

Extract public GeoJSON features with geometry type, bounds and selected non-contact properties.

Inventory public form control names, labels, types and constraints without submitting forms.

Extract repeating XML elements into child-field arrays and safe attributes without executing XML declarations.