Pricing

from $2.00 / 1,000 results

Go to Apify Store

Common Crawl Indexes Scraper

Try for free

Collect Common Crawl records and export id, name, to, from, cdx api, timegate as structured JSON, CSV or Excel data.

Pricing

from $2.00 / 1,000 results

Rating

0.0

(0)

Developer

Ben

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

Developer tools

Business

Automation

Apify Store pricing discounts use a tiered approach. Higher subscription plans unlock lower Actor costs. See our subscription plans.

Pricing	Free No discount	Starter Bronze discount	Scale Silver discount	Business Gold discount

Result	$2.50 / 1,000	$2.25 / 1,000	$2.13 / 1,000	$2.00 / 1,000
Actor start Infrequent	$0.00005	$0.000045	$0.000043	$0.00004
Additional costs
Platform usage	Free	Free	Free	Free

This Actor's pricing details are outlined in the table below. The unit prices you pay depend on your Apify subscription plan, with higher tiers offering better rates. Visit our pricing page to compare plans and see how much you can save.

Pricing	Free No discount	Business Gold discount

Result Charged for each normalized Common Crawl record saved to the default dataset.	$2.50 / 1,000	$2.00 / 1,000
Actor startInfrequent Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event).	$0.00005	$0.00004
Additional costs
Platform usageIncluded Platform usage costs are tied to the Actor's performance in terms of the underlying platform resources it consumes. Learn more	Free

Common Crawl URL Index Lookup Scraper

parseforge/common-crawl-index-scraper

Pull every web page Common Crawl captured for a domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and WARC offsets to fetch original payloads. Filter by collection, MIME, and status. Export to JSON, CSV, or Excel for large-scale web research and content discovery.

ParseForge

Ahrefs Free Tools Scraper

parseforge/ahrefs-tools-scraper

Pull SEO metrics from Ahrefs free tools: Domain Rating, backlinks, referring domains, organic traffic, keyword difficulty, and search volume. Submit a domain or keyword and get structured results. Export to JSON, CSV, or Excel for SEO research.

ParseForge

Wikipedia Pageviews Scraper

parseforge/wikipedia-pageviews-scraper

Pull Wikipedia pageview metrics for any article in any language edition. Daily or monthly granularity, filter by access type (desktop, mobile, app) and agent type (user, spider, automated). Pick a date range. Export to JSON, CSV, or Excel for SEO research and content benchmarking.

ParseForge

Common Crawl Scraper

crawlerbros/common-crawl-scraper

Query the Common Crawl URL Index for any domain or URL pattern. Discover a site's archived pages, historical URLs, capture dates, HTTP statuses and MIME types for SEO, domain intelligence and research. Also lists the available monthly crawls.

Crawler Bros

Shopify Store Finder — Target by Size & Tech Stack

cool_jacquard/shopify-store-enrichment

Find Shopify stores by installed apps (Klaviyo, Recharge, Gorgias, Yotpo, Loox), product count and country. Targeted B2B lead generation: enriched, revenue-qualified Shopify leads with email and socials — not a raw domain dump. Pay only for stores matching your filters.

Pavel Bragin

5.0

(2)

Find Companies Using Greenhouse, Lever, Ashby & Workable

wickfeed/ats-company-discovery

Find companies using Greenhouse, Ashby, Workable, Recruitee or Lever (EU) and see who's hiring right now — live open-role counts across ~1,990 companies. Filter by keyword. $2 per 1,000 companies.

WickFeed

Common Crawl Corpus Builder

zentrafoundry/common-crawl-targeted-corpus-builder

Build targeted text corpora from Common Crawl with provenance.

Zentra

Ultimate URL Harvester — All Site URLs from 6 Archives

inexhaustible_glass/url-harvester

Get every known URL of any domain from Wayback Machine, Common Crawl, AlienVault OTX, URLScan, crt.sh and sitemap — merged & deduped. Block-proof, no API key.

Hitman studio

Technical SEO Audit & Website Crawler - Core Web Vitals

groupoject/technical-seo-audit-crawler

Crawl websites and detect technical SEO issues across metadata, indexability, headings, content, links, images, structured data, security headers, and optional PageSpeed Core Web Vitals.

Group Oject

Web Images Scraper

maximedupre/web-images-scraper

Extract image URLs from public webpages, domains, and direct image links. Get source pages, discovery methods, metadata, and optional saved files or ZIP archives.