Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome) avatar

Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome)

Pricing

from $0.70 / 1,000 results

Go to Apify Store
Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome)

Web Unblocker - Anti-Bot Scraper (Cloudflare, DataDome)

Fetch any URL even behind anti-bot walls (Cloudflare, DataDome, PerimeterX, Akamai, Imperva). Returns HTML plus optional Markdown/text and a screenshot, detects which protection guards the site, and retries via a hardened browser on a fresh residential IP when blocked. No API key.

Pricing

from $0.70 / 1,000 results

Rating

0.0

(0)

Developer

Get Anything

Get Anything

Maintained by Community

Actor stats

0

Bookmarked

141

Total users

86

Monthly active users

9 days ago

Last modified

Categories

Features

Limited permissions

Share

Web Unblocker — Anti-Bot Scraper (Cloudflare, DataDome, PerimeterX, Akamai)

Fetch any URL and get the page back, even when it sits behind an anti-bot wall. Point it at a URL and it returns the final HTML (and optionally clean Markdown/text and a screenshot), tells you which protection guards the site, and retries through a hardened browser on a fresh IP when it gets blocked.

Built for the most common complaint in web scraping: "I'm blocked from this website, what are my options?"

How it works

  1. Fast path — an HTTP request with curl_cffi (real Chrome TLS/JA3 impersonation). Cheap and instant for unprotected pages.
  2. Browser render — if the fast path is blocked or challenged, the page is rendered in Camoufox (a hardened Firefox with humanized fingerprints) which clears most JavaScript anti-bot challenges from Apify IPs.
  3. Retry on a new IP — if it's still blocked, it retries on a fresh browser context with a new residential proxy IP, up to maxRetries times.

Anti-bot detection

Every result tells you what was protecting the site, matched from response headers + body fingerprints:

  • Cloudflare (cf-ray, "Just a moment", challenge-platform)
  • DataDome (x-datadome, captcha-delivery)
  • PerimeterX / HUMAN (_px* cookies, px-captcha)
  • Akamai Bot Manager (_abck, ak_bmsc)
  • Imperva / Incapsula (incap_ses, x-iinfo)
  • Generic CAPTCHA (reCAPTCHA / hCaptcha)

What you get

FieldDescription
url / finalUrlRequested URL and the URL after redirects
successWhether a real (non-challenge) page was returned
statusCodeHTTP status of the final response
protectionDetectedAnti-bot vendor detected, or null
bypassedtrue when a protection was present and cleared
methodhttp or browser
htmlFull page HTML (when output includes HTML)
markdown / textCleaned content with boilerplate removed (optional)
wordCountWord count of cleaned text (optional)
screenshotUrlLink to a full-page JPEG in the key-value store (optional)
scrapedAtISO 8601 timestamp

Input

{
"startUrls": [{ "url": "https://example.com/protected" }],
"renderJs": "auto",
"outputFormat": "html",
"waitForSelector": "",
"waitMs": 2000,
"screenshot": false,
"maxRetries": 2,
"proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}
FieldTypeDefaultDescription
startUrlsarray—URLs to fetch
renderJsstringautoauto (HTTP first, browser if blocked), always, or never
outputFormatstringhtmlhtml, markdown, text, or all
includeLinksbooleantrueKeep hyperlinks in Markdown
maxCharsinteger0Truncate Markdown/text (0 = no limit)
waitForSelectorstring—CSS selector to wait for (browser render)
waitMsinteger2000Wait after load when no selector (browser render)
screenshotbooleanfalseSave a full-page screenshot
maxRetriesinteger2Retries on a fresh IP when blocked
proxyConfigurationobjectResidentialProxy — residential strongly recommended

Use cases

  • Unblock a site you keep getting 403/429 on and parse the HTML yourself.
  • Check what anti-bot a site runs before you invest in building a scraper.
  • LLM/RAG ingestion of pages that need JS rendering, as clean Markdown.
  • Monitoring & QA of protected pages with screenshots.

FAQ

Can it bypass Cloudflare? Often, not always. When the fast HTTP request is challenged, the page is rendered in Camoufox (a hardened Firefox), which clears most JavaScript challenges, and is retried on a fresh residential IP up to maxRetries times. Each result says whether it worked in success and bypassed.

Which anti-bot systems does it detect? Cloudflare, DataDome, PerimeterX / HUMAN, Akamai Bot Manager, Imperva / Incapsula and generic reCAPTCHA / hCaptcha pages. The name goes in protectionDetected.

Does it solve CAPTCHAs? No. It detects reCAPTCHA and hCaptcha pages and reports them, but it does not run a CAPTCHA solver. It gets past JavaScript challenges by rendering the page in a real browser and rotating the IP.

Can I get the page as Markdown for an LLM? Yes. Set outputFormat to markdown, text or all to get cleaned content with boilerplate removed, plus wordCount. Use maxChars to cap the length.

Do I need my own proxy or API key? No. It uses Apify residential proxy by default and needs no API key. You can pass your own proxy in proxyConfiguration.

Can it fetch pages behind a login or paywall? No. It fetches publicly accessible pages only.

Use with Claude via Apify MCP

Give Claude (or ChatGPT, Cursor or any MCP client) a way to read pages that block normal fetches, through the Apify MCP server, with no code.

  1. In Claude: Settings → Connectors → Add custom connector → https://mcp.apify.com, then sign in to Apify.
  2. Ask in plain words, for example:
  3. Claude runs Web Unblocker with your URL and reads back the HTML or Markdown, the detected protection and whether the block was cleared.

To expose only this tool, use:

{ "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=get_anything/web-unblocker" } } }

Full guide: Apify MCP server documentation.

Notes

  • Only publicly accessible pages are fetched; no logins or paywalled content.
  • Residential proxy is what makes retries effective — each retry rotates to a new IP.
  • For maximum bypass reliability set renderJs to always.

Actors that use Web Unblocker

  • GCC Jobs Aggregator: UAE, Saudi and Gulf jobs; uses Web Unblocker to render Bayt and GulfTalent pages behind Cloudflare

⭐ Like this Actor?

If it saved you time, please leave a quick rating. It helps other people find it, and it keeps it maintained. Found a bug or need a field? Open an issue and I'll respond quickly.