Pricing

$29.00 / 1,000 result delivereds

Go to Apify Store

Common Crawl Corpus Builder

Try for free

Build targeted text corpora from Common Crawl with provenance.

Pricing

$29.00 / 1,000 result delivereds

Rating

0.0

(0)

Developer

Zentra

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Categories

Developer tools

Integrations

Regulatory Status Signal query

buyerQuery

Optional

Buyer-safe query or scope for the records returned by this product.

Type:string

Default:buyer-ready records with source evidence

Regulatory Status Signal task intent

taskIntent

Optional

Stable product-specific purpose for this saved Apify task.

Type:string

Default:buyer-ready-product-run

Run sample or approved live sources

sourceMode

Required

Sample emits public-safe Regulatory Status Signal rows. Approved live source mode keeps the same output fields and only uses owner-approved public URLs.

Type:string

Default:sample

Options:

samplestartUrls

Regulatory Status Signal output mode

outputMode

Optional

Use sample records for Apify Store QA or buyer-ready records for approved Regulatory Status Signal delivery.

Type:string

Default:sample-records

Options:

sample-recordsbuyer-ready-records

Approved Regulatory Status Signal source URLs

startUrls

Optional

Public URLs to use for live Regulatory Status Signal extraction after source-policy approval. Leave empty for sample mode.

Type:object[]

Default:

[]

Source request timeout seconds

requestTimeoutSecs

Optional

Timeout applied independently to each approved source request.

Type:integer

Minimum:5

Maximum:120

Default:30

Maximum source retries

maxRequestRetries

Optional

Bounded retry count for transient source failures.

Type:integer

Minimum:0

Maximum:5

Default:2

Only emit new logical records

sinceLastRun

Optional

Uses stable Actor state to skip logical records delivered by earlier runs.

Type:boolean

Default:false

Delta mode

deltaMode

Optional

Preserves stable deduplication keys for recurring tasks and schedules.

Type:boolean

Default:true

Common Crawl Indexes Scraper

benthepythondev/common-crawl-indexes-scraper

Collect Common Crawl records and export id, name, to, from, cdx api, timegate as structured JSON, CSV or Excel data.

Ben

Research Corpus & Citation Graph Builder

zentrafoundry/openalex-research-graph-builder

Build research corpora and citation graph datasets from public metadata APIs.

Zentra

Common Crawl Scraper

crawlerbros/common-crawl-scraper

Query the Common Crawl URL Index for any domain or URL pattern. Discover a site's archived pages, historical URLs, capture dates, HTTP statuses and MIME types for SEO, domain intelligence and research. Also lists the available monthly crawls.

Crawler Bros

Common Crawl URL Index Lookup Scraper

parseforge/common-crawl-index-scraper

Pull every web page Common Crawl captured for a domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and WARC offsets to fetch original payloads. Filter by collection, MIME, and status. Export to JSON, CSV, or Excel for large-scale web research and content discovery.

ParseForge

Actor Builder

handleco-app/actor-builder

handleco-app

Website Content Extractor

glowing_glove/website-content-extractor

Crawl public pages and extract page titles, meta descriptions, headings, readable text, source URLs, and crawl metadata.

Ushba Khan

Crawl4ai

kael_odin/crawl4ai

Extract page content (markdown/HTML/text), metadata, and link stats. Uses crawl4ai.

Kael Odin

Prophed Common Markets

prophed/prophed-common-markets

Fetch cross-platform prediction-market matches from Prophed Common Markets and output normalized arbitrage, liquidity, spread, and platform-comparison rows.

Prophed Com

Torrent / BitTorrent DHT corpus search & crawl statistics

apricot_inference/fast-tracker

Search a live BitTorrent DHT metadata corpus and see how fast it is growing. Two endpoints: /search with category filters and relevance ranking, and /stats with a per-minute capture rate you can verify. No endpoint or key needed — the corpus is crawled and hosted by this Actor's operator.