Web Content Extractor

Apify Actor for no-code friendly web content extraction. Fetches pages without proxy by default (cheap HTTP), uses Apify Proxy and Playwright only as fallback.

Features

• HTTP-first — direct Cheerio fetch without proxy (no proxy cost on most pages)
• 4-step fallback — Cheerio → Cheerio+Proxy → Playwright → Playwright+Proxy
• Clean content — Mozilla Readability removes nav, banners, and noise
• SEO metadata — title, meta description, OG tags, canonical, all headings (h1–h6)
• No-code output — flat JSON fields for Make, n8n, Zapier
• Social URL blocking — Facebook, YouTube, etc. return blocked_social without scraping

Input

Field	Default	Description
urls	—	Required. List of page URLs
outputFormat	json	json | text | markdown
useProxyFallback	true	Retry with Apify Proxy on HTTP errors / blocks
usePlaywrightFallback	true	Retry thin/JS pages with headless Chrome
minContentLength	200	Min text length before Playwright fallback
country	US	Proxy country (fallback steps only)
proxyType	RESIDENTIAL	RESIDENTIAL or DATACENTER
batchSize	50	URLs per processing wave
maxUrls	1000	Hard cap per run
extractExternalLinks	false	One URL per external domain

Example input

{
  "urls": [
    "https://example.com/article",
    "https://blog.example.org/post"
  ],
  "outputFormat": "json"
}

Output

One dataset item per URL.

Success example (outputFormat: json)

{
  "url": "https://example.com/article",
  "success": true,
  "status": "success_static",
  "method": "cheerio",
  "usedProxy": false,
  "title": "Article title",
  "metaDescription": "SEO description from meta tag",
  "ogTitle": "Article title",
  "ogDescription": "SEO description",
  "canonical": "https://example.com/article",
  "headings": [
    { "level": 1, "text": "Main heading" },
    { "level": 2, "text": "Section" }
  ],
  "fullText": "Clean plain text of the article...",
  "content": "Clean plain text of the article...",
  "markdown": "# Main heading\n\nClean plain text...",
  "wordCount": 842,
  "httpStatus": 200,
  "fetchedAt": "2026-06-24T12:00:00.000Z",
  "billable": true
}

outputFormat behavior

Format	content field	Other fields
json	Plain text (fullText)	All SEO fields + markdown
text	Plain text	title, success only (SEO fields null)
markdown	Markdown	All SEO fields

Status values

Status	Meaning
success_static	Cheerio HTTP fetch succeeded
success_rendered	Playwright rendering succeeded
blocked_social	Social URL — not scraped
failed_fetch	HTTP or network error
failed_dynamic	Content too short / not extractable
failed_timeout	Request timed out

Fallback chain

URLs → Cheerio (no proxy)
     → [403/block/timeout] → Cheerio (Apify Proxy)
     → [thin content]      → Playwright (no proxy)
     → [still failing]     → Playwright (Apify Proxy)

Proxy is initialized lazily — only when a fallback step needs it.

No-code integration

Make (Integromat)

1. Add Apify module → Run Actor → web-content-extractor
2. Pass urls array
3. Map output fields directly:
   • content → text field in Google Docs / Notion
   • metaDescription → SEO field
   • headings → iterator for outline generation
4. Filter: success = true

n8n

1. Apify node → Run Actor
2. Use JSON output — no markdown parsing needed
3. Expression: {{ $json.content }}, {{ $json.metaDescription }}
4. IF node: {{ $json.success }} equals true

Zapier

1. Apify action → Run Actor
2. Filter Zap: only continue if success is true
3. Map content and title to next step (e.g. OpenAI, Airtable)

Local development

npm install
npm run build
npm run dev

Set input via Apify CLI or storage/key_value_stores/default/INPUT.json:

{
  "urls": ["https://example.com"]
}

Docker

Built on apify/actor-node-playwright-chrome for Playwright fallback support.

$apify push

Billing

Pay-per-event: one scraped-url charge per successfully extracted URL. Proxy and Playwright fallback steps do not add extra charges.