Cloudflare / DataDome Bypass — Protected Page Change Monitor

Fetch pages that block ordinary scrapers — Cloudflare, Akamai, DataDome, PerimeterX — extract just the part you care about with a CSS selector or a regex, and get told when it changes. The actor escalates automatically (plain request → datacenter proxy → residential proxy → Apify Web Unblocker → headless Chromium) and stops at the first tier that returns real content, so expensive tiers are used only when they are needed.

What you get (example output)

Real rows from the example dataset (apify.com home page and an Etsy search page):

Field	Example value	What it is
url / finalUrl	https://www.etsy.com/search?q=ceramic+mug	Requested URL and the URL after redirects
status	200	HTTP status of the successful fetch
title	Ceramic mug - Etsy	<title> of the returned page
tierUsed	unblocker	Which tier finally got through (none, datacenter, residential, unblocker)
fetchMode	http	http (Chrome TLS fingerprint) or browser (headless Chromium)
bytes	637583	Size of the response body
contentHash	c70e019614022613894c393eeb21853a	MD5 of the extracted content — the value change detection compares
text	Etsy Categorías Buscar ...	Visible text (or selector text), capped by textLimit
extracted	["Ofertas especiales", "Se envía desde", ...]	Selector or regex matches, in document order
blocked / matched	false / true	Whether every tier was blocked, and whether the selector/regex matched

In monitor mode rows also carry changeType, changedFields and previous. htmlKey points at the full HTML in the run's key-value store when includeHtml is on.

Use cases

• Competitor price and stock watch — point a selector at the price element of a Cloudflare-protected shop and get a webhook when the number moves.
• Policy, terms and pricing-page tracking — keep a hash and a diff of pages that legal or procurement needs to know about.
• Feeding a pipeline that keeps hitting "just a moment…" — use it as the fetch step for any one-off page your own scraper cannot open.

How it works

Each URL is tried on the cheapest tier first. If the response looks like a challenge or block page, the actor escalates one tier and retries, up to the Apify Web Unblocker and — in auto or browser fetch mode — headless Chromium. The tier that succeeded is reported in tierUsed. Content is reduced to the selector or regex matches (or the visible text), hashed into contentHash, and in monitor mode compared against the previous run of the same Task, so only pages whose extracted content actually changed come back.

Input

Field	Meaning	Default
urls	Pages to fetch, using the default selector / regex below	["https://www.apify.com/"]
targets	Per-URL rules: [{"url": "...", "selector": ".price", "regex": "\\$([\\d.]+)", "label": "pricing", "waitFor": ".loaded"}]	empty
tier	auto (escalate until unblocked) or a fixed tier: none, datacenter, residential, unblocker	auto
proxyCountry	ISO country code for the residential / unblocker exit, e.g. US, DE	empty
fetchMode	http (Chrome TLS fingerprint, fast), auto (headless only if every HTTP tier is blocked), browser	http
selector	CSS selector applied to every URL, e.g. .price, #stock, table.rates	empty
regex	Python regex applied to the (selected) text; group 1 or the whole match is returned	empty
headers	Extra request headers, e.g. {"Accept-Language": "de-DE"}	empty
timeout	Per-request timeout in seconds (the Web Unblocker always gets at least 120 s)	60
textLimit	Characters of extracted text kept per page	5000
includeHtml	Store the complete HTML in the run's key-value store (htmlKey in the row)	false
maxItems	Stop after this many pushed rows	50
mode	scrape = every page; monitor = only pages whose extracted content changed since the previous run of this Task	scrape
monitorKey	State key for monitor mode when not running as a saved Task	empty
webhookUrl	URL that receives a POST with the change summary in monitor mode	empty
telegramBotToken / telegramChatId	Optional Telegram destination for monitor-mode summaries	empty

Pricing

Event	Price
result	$0.005 per page fetched ($5 per 1,000)
monitor-check	$0.01 per monitor run
change	$0.005 per changed page

A page that stayed blocked on every tier is returned with blocked: true and counts as one result; proxy / Web Unblocker traffic is billed by Apify on top, only for the tiers actually used.

Why this actor

• You pay for the cheap tier when the cheap tier works — escalation stops at the first tier that returns real content, and tierUsed tells you which one that was.
• Five tiers in one actor, including the Apify Web Unblocker and a headless Chromium fallback, with no code to write.
• Extraction built in — CSS selector and regex per URL, so the row holds the price or the stock line instead of half a megabyte of HTML.
• Change detection on the extracted content, not the whole page, so rotating banners and timestamps do not produce false alerts when you use a selector.
• Blocked is reported, not hidden — blocked: true and RUN_SUMMARY in the key-value store say exactly why a page produced nothing.
• No login, no cookies, no account on the target site.

Limits

• No CAPTCHA solving and no logins. Sites that require a challenge to be solved in a real, persistent browser session stay blocked.
• Change detection compares extracted text; without a selector, a page with rotating content will look changed on every run.
• Headless Chromium (fetchMode: browser) needs more memory and time than the HTTP modes, and the actor's minimum memory is 1 GB because of it.
• The actor fetches pages you point it at — it does not crawl or follow links.

FAQ

Can this bypass Cloudflare? It escalates through datacenter and residential proxies to the Apify Web Unblocker and, if needed, headless Chromium, which is enough for the "just a moment…" interstitial on most sites. Sites requiring an interactive CAPTCHA stay blocked and are returned with blocked: true.

Does it work with DataDome, Akamai or PerimeterX? The same escalation applies; the Web Unblocker tier is what usually gets through. tierUsed in each row tells you what the site actually demanded.

How do I monitor a page for changes? Give the URL and a CSS selector, save the actor as a Task, set mode to monitor, add webhookUrl or the Telegram fields and schedule the Task. Each run returns only pages whose selected content changed, with previous attached.

Changelog

• 0.1: initial release — tier escalation, selector/regex extraction, monitor mode, headless fallback.

---

If this actor is useful, a review on its Apify Store page genuinely helps other buyers find it. Found a bug or need a field that is missing? Open a ticket on the Issues tab of the actor and it will be looked at.