Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome)
Pricing
from $0.70 / 1,000 results
Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome)
Fetch any URL even behind anti-bot walls (Cloudflare, DataDome, PerimeterX, Akamai, Imperva). Returns HTML plus optional Markdown/text and a screenshot, detects which protection guards the site, and retries via a hardened browser on a fresh residential IP when blocked. No API key.
Pricing
from $0.70 / 1,000 results
Rating
0.0
(0)
Developer
Get Anything
Maintained by CommunityActor stats
0
Bookmarked
141
Total users
86
Monthly active users
9 days ago
Last modified
Categories
Features
Share
Web Unblocker — Anti-Bot Scraper (Cloudflare, DataDome, PerimeterX, Akamai)
Fetch any URL and get the page back, even when it sits behind an anti-bot wall. Point it at a URL and it returns the final HTML (and optionally clean Markdown/text and a screenshot), tells you which protection guards the site, and retries through a hardened browser on a fresh IP when it gets blocked.
Built for the most common complaint in web scraping: "I'm blocked from this website, what are my options?"
How it works
- Fast path — an HTTP request with
curl_cffi(real Chrome TLS/JA3 impersonation). Cheap and instant for unprotected pages. - Browser render — if the fast path is blocked or challenged, the page is rendered in Camoufox (a hardened Firefox with humanized fingerprints) which clears most JavaScript anti-bot challenges from Apify IPs.
- Retry on a new IP — if it's still blocked, it retries on a fresh browser context with a new residential proxy IP, up to
maxRetriestimes.
Anti-bot detection
Every result tells you what was protecting the site, matched from response headers + body fingerprints:
- Cloudflare (
cf-ray, "Just a moment", challenge-platform) - DataDome (
x-datadome, captcha-delivery) - PerimeterX / HUMAN (
_px*cookies, px-captcha) - Akamai Bot Manager (
_abck,ak_bmsc) - Imperva / Incapsula (
incap_ses,x-iinfo) - Generic CAPTCHA (reCAPTCHA / hCaptcha)
What you get
| Field | Description |
|---|---|
url / finalUrl | Requested URL and the URL after redirects |
success | Whether a real (non-challenge) page was returned |
statusCode | HTTP status of the final response |
protectionDetected | Anti-bot vendor detected, or null |
bypassed | true when a protection was present and cleared |
method | http or browser |
html | Full page HTML (when output includes HTML) |
markdown / text | Cleaned content with boilerplate removed (optional) |
wordCount | Word count of cleaned text (optional) |
screenshotUrl | Link to a full-page JPEG in the key-value store (optional) |
scrapedAt | ISO 8601 timestamp |
Input
{"startUrls": [{ "url": "https://example.com/protected" }],"renderJs": "auto","outputFormat": "html","waitForSelector": "","waitMs": 2000,"screenshot": false,"maxRetries": 2,"proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }}
| Field | Type | Default | Description |
|---|---|---|---|
startUrls | array | — | URLs to fetch |
renderJs | string | auto | auto (HTTP first, browser if blocked), always, or never |
outputFormat | string | html | html, markdown, text, or all |
includeLinks | boolean | true | Keep hyperlinks in Markdown |
maxChars | integer | 0 | Truncate Markdown/text (0 = no limit) |
waitForSelector | string | — | CSS selector to wait for (browser render) |
waitMs | integer | 2000 | Wait after load when no selector (browser render) |
screenshot | boolean | false | Save a full-page screenshot |
maxRetries | integer | 2 | Retries on a fresh IP when blocked |
proxyConfiguration | object | Residential | Proxy — residential strongly recommended |
Use cases
- Unblock a site you keep getting 403/429 on and parse the HTML yourself.
- Check what anti-bot a site runs before you invest in building a scraper.
- LLM/RAG ingestion of pages that need JS rendering, as clean Markdown.
- Monitoring & QA of protected pages with screenshots.
FAQ
Can it bypass Cloudflare? Often, not always. When the fast HTTP request is challenged, the page is rendered in Camoufox (a hardened Firefox), which clears most JavaScript challenges, and is retried on a fresh residential IP up to maxRetries times. Each result says whether it worked in success and bypassed.
Which anti-bot systems does it detect? Cloudflare, DataDome, PerimeterX / HUMAN, Akamai Bot Manager, Imperva / Incapsula and generic reCAPTCHA / hCaptcha pages. The name goes in protectionDetected.
Does it solve CAPTCHAs? No. It detects reCAPTCHA and hCaptcha pages and reports them, but it does not run a CAPTCHA solver. It gets past JavaScript challenges by rendering the page in a real browser and rotating the IP.
Can I get the page as Markdown for an LLM? Yes. Set outputFormat to markdown, text or all to get cleaned content with boilerplate removed, plus wordCount. Use maxChars to cap the length.
Do I need my own proxy or API key? No. It uses Apify residential proxy by default and needs no API key. You can pass your own proxy in proxyConfiguration.
Can it fetch pages behind a login or paywall? No. It fetches publicly accessible pages only.
Use with Claude via Apify MCP
Give Claude (or ChatGPT, Cursor or any MCP client) a way to read pages that block normal fetches, through the Apify MCP server, with no code.
- In Claude: Settings → Connectors → Add custom connector →
https://mcp.apify.com, then sign in to Apify. - Ask in plain words, for example:
- "Fetch https://example.com/pricing as Markdown and summarise the plans."
- "Which anti-bot protection does this site use?"
- Claude runs Web Unblocker with your URL and reads back the HTML or Markdown, the detected protection and whether the block was cleared.
To expose only this tool, use:
{ "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=get_anything/web-unblocker" } } }
Full guide: Apify MCP server documentation.
Notes
- Only publicly accessible pages are fetched; no logins or paywalled content.
- Residential proxy is what makes retries effective — each retry rotates to a new IP.
- For maximum bypass reliability set
renderJstoalways.
Actors that use Web Unblocker
- GCC Jobs Aggregator: UAE, Saudi and Gulf jobs; uses Web Unblocker to render Bayt and GulfTalent pages behind Cloudflare
Related actors
- LLM Web Scraper - Website to Markdown for RAG & ChatGPT
- AI Web Search Scraper - SERP & Page Content for LLM RAG
- SERP Scraper - Google, Bing & DuckDuckGo for LLM/RAG
- YouTube Transcript Scraper - LLM-Ready Video Text
⭐ Like this Actor?
If it saved you time, please leave a quick rating. It helps other people find it, and it keeps it maintained. Found a bug or need a field? Open an issue and I'll respond quickly.