Wayback Machine CDX URL List Scraper

Wayback Machine Archive Scraper

andok/wayback-machine-scraper

Fetch historical snapshots of any webpage from the Internet Archive. Perfect for digital forensics and tracking deleted content.

Wayback Machine snapshots CSV/JSON — per snapshot: timestamp, status, MIME, size, archive URL — date-filterable. CDX API, no key. 21+ runs. For competitor history-tracking + SEO recovery + brand archaeology. spinov001@gmail.com · blog.spinov.online · t.me/scraping_ai

Wayback Machine Search

ryanclinton/wayback-machine-search

Search the Internet Archive's Wayback Machine for historical snapshots of any website. Retrieve archived page metadata -- including timestamps, URLs, MIME types, HTTP status codes, and content hashes -- for up to 10,000 snapshots per run.

Common Crawl URL Index Lookup Scraper

parseforge/common-crawl-index-scraper

Pull every web page Common Crawl captured for a domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and WARC offsets to fetch original payloads. Filter by collection, MIME, and status. Export to JSON, CSV, or Excel for large-scale web research and content discovery.

Federal Court Locator Scraper

parseforge/pacer-court-locator-scraper

Pull every US federal court (district, bankruptcy, appellate, special) with PACER court ID, FJC ID, RSS feed, court URL, jurisdiction code, citation string, and active status. Filter by jurisdiction, state, or court name. Export to JSON, CSV, or Excel for legal directories and court tracking.

Ahrefs Free Tools Scraper

parseforge/ahrefs-tools-scraper

Pull SEO metrics from Ahrefs free tools: Domain Rating, URL Rating, backlinks, referring domains, dofollow backlinks, organic traffic, and organic keywords. Switch between domain and keyword tools. Export to JSON, CSV, or Excel for SEO research, competitor benchmarking, and content audits.

California State Bar Scraper

parseforge/california-state-bar-scraper

Pull California attorney profiles from the public State Bar directory: full name, bar number, license status, address, phone, email, county, district, sections, practice areas, and law school. Search by name or bar number. Export to JSON, CSV, or Excel for legal research and verification.

Bing Search Results Scraper

parseforge/bing-search-scraper

Pull Bing organic search results with rank, title, URL, snippet, displayed URL, deep links, and date. Filter by country, market, language, safe-search, and freshness. Export clean JSON, CSV, or Excel for SEO research, competitor monitoring, content benchmarking, and SERP intelligence.

DuckDuckGo Search Results Scraper

parseforge/duckduckgo-search-scraper

Pull DuckDuckGo organic search results with rank, title, URL, snippet, displayed URL, and favicon. Filter by region, time range, and safe-search. Export clean JSON, CSV, or Excel for privacy-respecting SEO research, competitor discovery, and SERP monitoring with no rate-limit headaches.

Shopify App Store Scraper

parseforge/shopify-app-store-scraper

Pull Shopify apps with name, developer, rating, review count, category, description, screenshots, and icon. Discover via sitemap with 30,000+ apps indexed. Filter by keyword to narrow the results. Export to JSON, CSV, or Excel for ecommerce ecosystem research and competitive analysis.