Website Content Extractor

Web page URLs in, clean readable content out.

What it does

Extracts readable paragraph text from web pages. It removes script/style/nav/header/footer-type boilerplate, prefers the first <main> or <article> region, and keeps <p> paragraphs of at least 4 words. Returns structured JSON with the page title, meta description (falls back to og:description / twitter:description), clean text, word count, and the language declared in <html lang> (or og:locale). Pages whose text is not in <p> paragraphs can yield no readable content; those are reported as errors (see below).

Deterministic HTML parsing. No AI models. No browser automation. Fast and cheap.

Input

{
  "pageUrls": [
    "https://example.com/article-1",
    "https://example.com/article-2"
  ]
}

Output

{
  "input": "https://example.com/article-1",
  "url": "https://example.com/article-1",
  "finalUrl": "https://example.com/article-1",
  "title": "Article Title",
  "description": "Meta description text",
  "text": "Clean article text without boilerplate...",
  "wordCount": 1247,
  "truncated": false,
  "language": "en",
  "status": "ok",
  "actorVersion": "0.1",
  "contractId": "WCE-CONTRACT-2026-10-05"
}

Successful records also carry page metadata fields (canonicalUrl, openGraph, twitterCard, meta, jsonLd, jsonLdTypes, h1). text is cut at maxChars (default 20,000, range 1,000–100,000) and truncated says whether it was. description is "" (empty string) when the page has no meta, og:description, or twitter:description tag — common on Wikipedia and many other pages.

status is "ok" or "error". An error record always has input, url, and error. A page that was fetched but produced 0 words of readable text is an error ("no readable content extracted (0 words)"), not "ok". The key-value store record run-summary holds requested, succeeded, failed, and billable_units.

Use cases

• Content aggregation pipelines
• Research data collection
• RAG preprocessing (clean text for embeddings)
• Web monitoring and change detection
• Article archiving

Pricing

$0.002646 per page extracted ($2.65 per 1,000 pages)

You are charged one page-extracted event only for status: "ok" records (readable content with at least one word). Error records (404 or other HTTP errors, blocked fetches, invalid URLs, no readable content) are not billed.

Limitations

• Static HTML only; JavaScript-rendered content is not supported
• Does not read or apply robots.txt; only submit URLs you are permitted to fetch
• No proxy or anti-bot bypass; blocked sites return error records
• Reads at most the first 2 MB of each page
• Paywalled content is not bypassed
• Non-HTML content (PDFs, images) returns an error

Failure behavior

One failed URL never kills the batch. Each page is processed independently; successful extractions are returned with explicit error records for failures.

API usage

curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pageUrls": ["https://example.com"]}'