Stealth Web Scraper & Cloudflare Bypass — Markdown Extractor

Fast web fetching by default. Automatic Cloudflare/anti-bot bypass when blocked. Clean Markdown for AI, RAG & automation.

Fetch difficult, bot-protected web pages without sending every URL through a browser — and without paying stealth-browser prices for pages that didn't need one.

🎬 See it in action

Fast → blocked → automatic stealth → HTTP 200 → clean Markdown

The Actor starts with a lightweight fetch and automatically escalates to a stealth browser only when a site actually blocks it. You can also force stealth mode for sites known to require a browser.

---

Why use it?

⚡ Fast first, stealth only when needed

URL
 ↓
Fast HTTP fetch
 ↓
Works? ─────────────── Yes → Clean Markdown
 ↓ No
Stealth browser (Cloudflare/anti-bot bypass)
 ↓
Clean Markdown

Verified results from real Actor runs:

Task	Result	Tier used	Extraction
example.com	HTTP 200	fast	title → "Example Domain"
MDN	HTTP 200	fast	title → "Add JavaScript to your web page"
Cloudflare Challenge page	HTTP 200	stealth	title → "Cloudflare Challenge"

Every row above is a successful extraction — the Cloudflare-protected page returned a real HTTP 200 via automatic stealth escalation, not a placeholder or partial result.

You can also set forceStealth: true for sites already known to require a browser.

---

💰 Pricing

Free tier, forever: 300 fast-tier pages and 25 stealth-tier pages every month, on every account — enough to fully evaluate this Actor before spending anything.

What triggers a charge	Price	Free allowance / month
Actor start	$0.005	—
Fast-tier page (no anti-bot resistance needed)	$0.005	300
Stealth-tier page (Cloudflare/anti-bot escalation)	$0.035	25
Structured field extracted (optional, via extractionSchema)	$0.002	—

You're only ever charged the stealth price for pages that actually needed it — a page resolved on the fast tier is never billed at the stealth rate.

Example volume (typical mix)	Estimated cost
1,000 pages, ~80% fast / 20% stealth	~$11.00
10,000 pages, ~80% fast / 20% stealth	~$110.00

---

🧹 Clean, LLM-ready Markdown

Turn web pages into useful Markdown instead of feeding your AI pipeline raw HTML, navigation, and page chrome.

The Markdown output preserves headings, paragraphs, links, lists/tables, and code blocks — suitable for RAG pipelines, embeddings, AI agents, research, content monitoring, and knowledge-base ingestion.

---

🎯 Optional structured extraction

Request additional fields using simple CSS selectors:

[
  { "name": "title", "selector": "h1" },
  { "name": "author", "selector": "[rel='author']" }
]

{
  "extracted": {
    "title": "Example Domain",
    "author": "John Doe"
  }
}

Adaptive (self-healing) extraction

Fields extracted via extractionSchema use Scrapling's adaptive-relocation mechanism: when the exact selector stops matching, the field is relocated by DOM similarity instead of returning null. Verified end-to-end in Showcase 5, including the actual similarity score behind a real recovery (100% match). Still validate adaptive matching against your own target sites before relying on it unattended in production -- one verified site is a proof of the mechanism, not a guarantee across every possible page structure.

---

🚀 Quick start

{ "startUrls": [{ "url": "https://example.com" }] }

Force stealth for a site known to require a browser:

{ "startUrls": [{ "url": "https://example.com" }], "forceStealth": true }

With structured extraction:

{
  "startUrls": [{ "url": "https://example.com" }],
  "extractionSchema": [{ "name": "title", "selector": "h1" }]
}

---

🧪 Showcases

Each showcase includes the copyable input and the actual output from a real Actor run, so you can compare your own run against it.

1. Normal page → fast fetch

Input

{
  "startUrls": [{ "url": "https://example.com" }],
  "extractionSchema": [{ "name": "title", "selector": "h1" }],
  "forceStealth": false,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": false }
}

Output

{
  "url": "https://example.com",
  "success": true,
  "statusCode": 200,
  "fetchTier": "fast",
  "extracted": { "title": "Example Domain" }
}

2. MDN → fast fetch + structured extraction

Input

{
  "startUrls": [{ "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/How_to/Add_JavaScript_to_your_web_page" }],
  "extractionSchema": [{ "name": "title", "selector": "h1" }],
  "forceStealth": false,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": false }
}

Output

{
  "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/How_to/Add_JavaScript_to_your_web_page",
  "success": true,
  "statusCode": 200,
  "fetchTier": "fast",
  "extracted": { "title": "Add JavaScript to your web page" }
}

3. Cloudflare Challenge → automatic stealth escalation

Input

{
  "startUrls": [{ "url": "https://www.scrapingcourse.com/cloudflare-challenge" }],
  "extractionSchema": [{ "name": "title", "selector": "h1" }],
  "forceStealth": false,
  "markdownThreshold": 45,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": false }
}

Output

{
  "url": "https://www.scrapingcourse.com/cloudflare-challenge",
  "success": true,
  "statusCode": 200,
  "fetchTier": "stealth",
  "extracted": { "title": "Cloudflare Challenge" },
  "markdown": "# Cloudflare Challenge\n## You bypassed the Cloudflare challenge! :D"
}

Note on proxy settings above: examples 1–3 deliberately run with useApifyProxy: false to demonstrate the escalation ladder working even in the hardest case — no proxy at all. For production use, set useApifyProxy: true (the input schema default) — see Example 4.

4. Recommended production config (residential proxy)

Input

{
  "startUrls": [{ "url": "https://www.scrapingcourse.com/cloudflare-challenge" }],
  "extractionSchema": [{ "name": "title", "selector": "h1" }],
  "forceStealth": false,
  "markdownThreshold": 45,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": true }
}

Output

{
  "url": "https://www.scrapingcourse.com/cloudflare-challenge",
  "success": true,
  "statusCode": 200,
  "fetchTier": "stealth",
  "extracted": { "title": "Cloudflare Challenge" },
  "markdown": "# Cloudflare Challenge\n## You bypassed the Cloudflare challenge! :D"
}

Same successful result as Example 3, confirmed with the recommended proxy setting.

5. Self-healing in action -- verified

Two-run test: Run A seeds a fingerprint with a working selector, Run B repeats the same URL and field name with a deliberately broken selector that matches nothing on the page.

Run A -- seed a fingerprint

{
  "startUrls": [{ "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/How_to/Add_JavaScript_to_your_web_page" }],
  "extractionSchema": [{ "name": "heading", "selector": "h1" }],
  "forceStealth": false,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": true }
}

{ "success": true, "statusCode": 200, "fetchTier": "fast", "extracted": { "heading": "Add JavaScript to your web page" } }

Run B -- same URL/field, selector changed to h1.this-class-no-longer-exists

{
  "startUrls": [{ "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/How_to/Add_JavaScript_to_your_web_page" }],
  "extractionSchema": [{ "name": "heading", "selector": "h1.this-class-no-longer-exists" }],
  "forceStealth": false,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 60,
  "proxyConfiguration": { "useApifyProxy": true }
}

{ "success": true, "statusCode": 200, "fetchTier": "fast", "extracted": { "heading": "Add JavaScript to your web page" } }

The field was recovered even though the selector matched nothing on the page. This isn't just a pass/fail result -- the run log shows the actual relocation mechanism at work:

DEBUG: Highest probability was 100.0%
DEBUG: Top 5 best matching elements are:
100.0 -> [<data='<h1>Add JavaScript to your web page</h1>' parent='<div class="layout__header...'>]
62.24 -> [<data='<mdn-survey></mdn-survey>' ...>]
49.28 -> [<data='<p> Your blueprint for a better internet...' ...>]

The correct <h1> scored a perfect 100% match against the fingerprint saved in Run A, comfortably ahead of the next-best candidate (62%) -- a clean, unambiguous relocation, not a lucky guess near the threshold.

6. Graceful failure handling

Input

{
  "startUrls": [{ "url": "https://this-domain-does-not-exist-abcxyz123.com" }],
  "extractionSchema": [],
  "forceStealth": false,
  "maxConcurrency": 1,
  "maxUrlsPerRun": 1,
  "fetchTimeoutSecs": 30,
  "proxyConfiguration": { "useApifyProxy": true }
}

Output

{
  "url": "https://this-domain-does-not-exist-abcxyz123.com",
  "success": false,
  "error": "Error: Page.goto: net::ERR_TUNNEL_CONNECTION_FAILED at https://this-domain-does-not-exist-abcxyz123.com/"
}

The run completed normally with success: false and a real, actionable error message — it did not crash or block other URLs in the same run.

A failed or timed-out URL is recorded with success: false and an error message — it never stops the rest of a multi-URL run.

---

📤 Output

Each URL produces a result containing url, fetchedAt, success, statusCode, fetchTier (fast or stealth), markdown, extracted, and — only on failure — error.

{
  "url": "https://example.com",
  "fetchedAt": "2026-08-16T14:21:22.097540+00:00",
  "success": true,
  "statusCode": 200,
  "fetchTier": "fast",
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples...",
  "extracted": {
    "title": "Example Domain",
    "paragraph1": "This domain is for use in documentation examples..."
  }
}

---

When should I use this?

Good fit

• AI / RAG — ingest documentation, public information, and web content
• AI agents — provide agents with clean web-page content
• Research — collect readable content from multiple websites
• Competitive intelligence — monitor pages that may be difficult to fetch
• SEO & agencies — recurring content and page audits
• Sales & RevOps — collect public company and product information
• Automation — use the results in n8n, LangChain, CrewAI, MCP, or custom workflows

When to use something else

If you need...	Consider instead
Login-only / account-gated pages	A custom Actor with session/auth handling — this Actor only accesses publicly reachable pages
Structured data from a specific major site (LinkedIn, Amazon, etc.)	A dedicated site-specific Actor — usually cheaper and more structured than generic CSS selectors
Guaranteed success on every request	No tool can promise this — sites change and anti-bot systems evolve
Pages with no anti-bot protection at all	A simpler/cheaper HTTP-only crawler may be more cost-effective than paying for stealth-tier headroom you won't use

---

❓ FAQ

Do I need my own proxy? No — Apify Proxy is used by default and is strongly recommended, especially for the stealth tier (see Example 4). You can supply your own proxy if you have an existing contract.

Is this free to try? Yes — every account gets 300 free fast-tier pages and 25 free stealth-tier pages every month, permanently, not just during a trial window.

What happens if a page fails or times out? It's recorded with success: false and an error field (see Showcase 6). The rest of the run continues normally.

Will this work on any Cloudflare-protected site? The stealth tier is built to solve Cloudflare-style managed and embedded challenges automatically. Results can vary by target — validate against your specific site before relying on it for unattended production jobs.

Is the "self-healing" claim real, or marketing? Real and verified -- see Showcase 5 for a reproducible before/after test, including the actual relocation confidence score (100% match) from a real run log, not just a claimed pass/fail. It's not infallible on every possible page: validate it against your own target sites, especially pages with many visually similar elements.

---

⚙️ Input

Field	Purpose
startUrls	URLs to fetch
extractionSchema	Optional {name, selector} fields, matched adaptively
forceStealth	Skip directly to the stealth tier
markdownThreshold	Markdown boilerplate pruning aggressiveness
maxConcurrency	Parallel URL limit
maxUrlsPerRun	Maximum URLs processed per run
fetchTimeoutSecs	Per-URL timeout
proxyConfiguration	Apify Proxy (recommended) or your own proxy

---

🧩 Built for AI and automation

Web page → Clean Markdown → RAG / embeddings / AI agent / automation

Compatible with n8n, LangChain, CrewAI, MCP, and custom Python/JavaScript workflows.

---

Open source

Markdown generation uses Crawl4AI (Apache 2.0).

This product includes software developed by UncleCode (https://x.com/unclecode) as part of the Crawl4AI project (https://github.com/unclecode/crawl4ai).

Stealth fetching and adaptive extraction use Scrapling (BSD-3-Clause), with Camoufox (MPL-2.0) powering the stealth-browser tier.

---

Acceptable use

You are responsible for complying with each target site's Terms of Service, robots.txt, applicable laws, and Apify's Acceptable Use Policy. This Actor provides a general-purpose fetching capability — use stealth features only where you have the right to access the target content.

---

Try it

Give it a normal URL and let it stay fast. Give it a difficult or protected URL and let it escalate when necessary.

Fast when possible. Stealth when needed. Clean Markdown every time.