# Cloudflare / DataDome Bypass — Protected Page Change Monitor (`datahamster/protected-page-fetch-monitor`) Actor

Cloudflare bypass and fetcher for pages behind Cloudflare, Akamai, DataDome or PerimeterX: automatic escalation (plain → datacenter → residential → Web Unblocker → headless Chromium), CSS-selector or regex extraction, and change alerts by webhook or Telegram. Pay per page fetched, no login.

- **URL**: https://apify.com/datahamster/protected-page-fetch-monitor.md
- **Developed by:** [Viktor Dubnytskiy](https://apify.com/datahamster) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 page fetcheds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Cloudflare / DataDome Bypass — Protected Page Change Monitor

Fetch pages that block ordinary scrapers — Cloudflare, Akamai, DataDome, PerimeterX — extract just the part you care about with a CSS selector or a regex, and get told when it changes. The actor escalates automatically (plain request → datacenter proxy → residential proxy → Apify Web Unblocker → headless Chromium) and stops at the first tier that returns real content, so expensive tiers are used only when they are needed.

### What you get (example output)

Real rows from the example dataset (`apify.com` home page and an Etsy search page):

| Field | Example value | What it is |
|---|---|---|
| `url` / `finalUrl` | `https://www.etsy.com/search?q=ceramic+mug` | Requested URL and the URL after redirects |
| `status` | `200` | HTTP status of the successful fetch |
| `title` | `Ceramic mug - Etsy` | `<title>` of the returned page |
| `tierUsed` | `unblocker` | Which tier finally got through (`none`, `datacenter`, `residential`, `unblocker`) |
| `fetchMode` | `http` | `http` (Chrome TLS fingerprint) or `browser` (headless Chromium) |
| `bytes` | `637583` | Size of the response body |
| `contentHash` | `c70e019614022613894c393eeb21853a` | MD5 of the extracted content — the value change detection compares |
| `text` | `Etsy Categorías Buscar ...` | Visible text (or selector text), capped by `textLimit` |
| `extracted` | `["Ofertas especiales", "Se envía desde", ...]` | Selector or regex matches, in document order |
| `blocked` / `matched` | `false` / `true` | Whether every tier was blocked, and whether the selector/regex matched |

In monitor mode rows also carry `changeType`, `changedFields` and `previous`. `htmlKey` points at the full HTML in the run's key-value store when `includeHtml` is on.

### Use cases

- **Competitor price and stock watch** — point a selector at the price element of a Cloudflare-protected shop and get a webhook when the number moves.
- **Policy, terms and pricing-page tracking** — keep a hash and a diff of pages that legal or procurement needs to know about.
- **Feeding a pipeline that keeps hitting "just a moment…"** — use it as the fetch step for any one-off page your own scraper cannot open.

### How it works

Each URL is tried on the cheapest tier first. If the response looks like a challenge or block page, the actor escalates one tier and retries, up to the Apify Web Unblocker and — in `auto` or `browser` fetch mode — headless Chromium. The tier that succeeded is reported in `tierUsed`. Content is reduced to the selector or regex matches (or the visible text), hashed into `contentHash`, and in `monitor` mode compared against the previous run of the same Task, so only pages whose extracted content actually changed come back.

### Input

| Field | Meaning | Default |
|---|---|---|
| `urls` | Pages to fetch, using the default `selector` / `regex` below | `["https://www.apify.com/"]` |
| `targets` | Per-URL rules: `[{"url": "...", "selector": ".price", "regex": "\\$([\\d.]+)", "label": "pricing", "waitFor": ".loaded"}]` | empty |
| `tier` | `auto` (escalate until unblocked) or a fixed tier: `none`, `datacenter`, `residential`, `unblocker` | `auto` |
| `proxyCountry` | ISO country code for the residential / unblocker exit, e.g. `US`, `DE` | empty |
| `fetchMode` | `http` (Chrome TLS fingerprint, fast), `auto` (headless only if every HTTP tier is blocked), `browser` | `http` |
| `selector` | CSS selector applied to every URL, e.g. `.price`, `#stock`, `table.rates` | empty |
| `regex` | Python regex applied to the (selected) text; group 1 or the whole match is returned | empty |
| `headers` | Extra request headers, e.g. `{"Accept-Language": "de-DE"}` | empty |
| `timeout` | Per-request timeout in seconds (the Web Unblocker always gets at least 120 s) | `60` |
| `textLimit` | Characters of extracted text kept per page | `5000` |
| `includeHtml` | Store the complete HTML in the run's key-value store (`htmlKey` in the row) | `false` |
| `maxItems` | Stop after this many pushed rows | `50` |
| `mode` | `scrape` = every page; `monitor` = only pages whose extracted content changed since the previous run of this Task | `scrape` |
| `monitorKey` | State key for monitor mode when not running as a saved Task | empty |
| `webhookUrl` | URL that receives a POST with the change summary in monitor mode | empty |
| `telegramBotToken` / `telegramChatId` | Optional Telegram destination for monitor-mode summaries | empty |

### Pricing

| Event | Price |
|---|---|
| result | $0.005 per page fetched ($5 per 1,000) |
| monitor-check | $0.01 per monitor run |
| change | $0.005 per changed page |

A page that stayed blocked on every tier is returned with `blocked: true` and counts as one result; proxy / Web Unblocker traffic is billed by Apify on top, only for the tiers actually used.

### Why this actor

- **You pay for the cheap tier when the cheap tier works** — escalation stops at the first tier that returns real content, and `tierUsed` tells you which one that was.
- **Five tiers in one actor**, including the Apify Web Unblocker and a headless Chromium fallback, with no code to write.
- **Extraction built in** — CSS selector and regex per URL, so the row holds the price or the stock line instead of half a megabyte of HTML.
- **Change detection on the extracted content**, not the whole page, so rotating banners and timestamps do not produce false alerts when you use a selector.
- **Blocked is reported, not hidden** — `blocked: true` and `RUN_SUMMARY` in the key-value store say exactly why a page produced nothing.
- **No login, no cookies, no account on the target site.**

### Limits

- No CAPTCHA solving and no logins. Sites that require a challenge to be solved in a real, persistent browser session stay blocked.
- Change detection compares extracted text; without a selector, a page with rotating content will look changed on every run.
- Headless Chromium (`fetchMode: browser`) needs more memory and time than the HTTP modes, and the actor's minimum memory is 1 GB because of it.
- The actor fetches pages you point it at — it does not crawl or follow links.

### FAQ

**Can this bypass Cloudflare?** It escalates through datacenter and residential proxies to the Apify Web Unblocker and, if needed, headless Chromium, which is enough for the "just a moment…" interstitial on most sites. Sites requiring an interactive CAPTCHA stay blocked and are returned with `blocked: true`.

**Does it work with DataDome, Akamai or PerimeterX?** The same escalation applies; the Web Unblocker tier is what usually gets through. `tierUsed` in each row tells you what the site actually demanded.

**How do I monitor a page for changes?** Give the URL and a CSS selector, save the actor as a Task, set `mode` to `monitor`, add `webhookUrl` or the Telegram fields and schedule the Task. Each run returns only pages whose selected content changed, with `previous` attached.

### Changelog

- 0.1: initial release — tier escalation, selector/regex extraction, monitor mode, headless fallback.

***

If this actor is useful, a review on its Apify Store page genuinely helps other buyers find it. Found a bug or need a field that is missing? Open a ticket on the **Issues** tab of the actor and it will be looked at.

# Actor input Schema

## `urls` (type: `array`):

Pages to fetch, as full URLs, e.g. \["https://www.apify.com/"]. Each URL becomes one row and uses the default selector/regex below.

## `targets` (type: `array`):

Per-URL extraction rules as JSON, e.g. \[{"url": "https://shop.example.com/p/1", "selector": ".price", "regex": "\\$(\[\d.]+)", "label": "pricing", "waitFor": ".loaded"}]. Use instead of urls when each page needs its own selector; label is echoed back in the row.

## `tier` (type: `string`):

Proxy tier: "auto" tries plain → datacenter → residential → Web Unblocker and stops at the first tier that is not blocked (recommended, and you pay Apify proxy traffic only for the tiers actually used). Fix it with "none", "datacenter", "residential" or "unblocker". Example: "auto".

## `proxyCountry` (type: `string`):

Two-letter ISO country code for the residential or Web Unblocker exit, e.g. "US" or "DE". Leave empty to let Apify choose; set it when the page is geo-dependent.

## `fetchMode` (type: `string`):

"http" sends plain requests with a Chrome TLS fingerprint (fast, cheap); "auto" falls back to headless Chromium only when every HTTP tier is blocked; "browser" always renders in headless Chromium (needed for JavaScript-built pages, slower). Example: "http".

## `selector` (type: `string`):

CSS selector applied to every URL in urls; only matching elements' text is kept, e.g. ".price", "#stock" or "table.rates". Leave empty to keep the page's visible text.

## `regex` (type: `string`):

Python regular expression applied to the (selected) text; capture group 1 is returned when present, otherwise the whole match, e.g. "\\$(\[0-9.]+)". Leave empty to skip regex extraction.

## `headers` (type: `object`):

Extra HTTP headers sent with every request, as a JSON object, e.g. {"Accept-Language": "de-DE"}.

## `timeout` (type: `integer`):

Per-request timeout in seconds, e.g. 60. The Web Unblocker tier always gets at least 120 s regardless of this value.

## `textLimit` (type: `integer`):

How many characters of extracted text to keep per page, e.g. 5000. Longer text is truncated.

## `includeHtml` (type: `boolean`):

If true, store the complete HTML of each page in the run's key-value store and put its key in the row's htmlKey field. Example: false.

## `maxItems` (type: `integer`):

Stop after this many pushed rows, e.g. 50. You are charged only for rows actually pushed; a page blocked on every tier still counts as one row.

## `mode` (type: `string`):

"scrape" returns a row for every page; "monitor" returns only pages whose extracted content changed since the previous run of the same saved Task, with changeType, changedFields and previous attached. Example: "scrape".

## `monitorKey` (type: `string`):

State key for monitor mode when the actor is not run as a saved Task, e.g. "competitor-pricing-page". Runs sharing a key share the comparison state.

## `webhookUrl` (type: `string`):

URL that receives a POST with the monitor-mode change summary (JSON), e.g. "https://hooks.zapier.com/hooks/catch/123/abc". Leave empty to skip.

## `telegramBotToken` (type: `string`):

Optional Telegram bot token used to send monitor-mode change summaries, e.g. "123456789:AAF...". Leave empty to skip Telegram.

## `telegramChatId` (type: `string`):

Optional Telegram chat id that receives monitor-mode summaries, e.g. "-1001234567890". Required only when telegramBotToken is set.

## Actor input object example

```json
{
  "urls": [
    "https://www.apify.com/"
  ],
  "tier": "auto",
  "fetchMode": "http",
  "timeout": 60,
  "textLimit": 5000,
  "includeHtml": false,
  "maxItems": 50,
  "mode": "scrape"
}
```

# Actor output Schema

## `results` (type: `string`):

All pushed rows (dataset, JSON)

## `resultsTable` (type: `string`):

Dataset in the Console viewer

## `runSummary` (type: `string`):

RUN\_SUMMARY record (pushed, skipped, emptyReason)

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://www.apify.com/"
    ],
    "tier": "auto",
    "fetchMode": "http",
    "timeout": 60,
    "textLimit": 5000,
    "maxItems": 50,
    "mode": "scrape"
};

// Run the Actor and wait for it to finish
const run = await client.actor("datahamster/protected-page-fetch-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://www.apify.com/"],
    "tier": "auto",
    "fetchMode": "http",
    "timeout": 60,
    "textLimit": 5000,
    "maxItems": 50,
    "mode": "scrape",
}

# Run the Actor and wait for it to finish
run = client.actor("datahamster/protected-page-fetch-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://www.apify.com/"
  ],
  "tier": "auto",
  "fetchMode": "http",
  "timeout": 60,
  "textLimit": 5000,
  "maxItems": 50,
  "mode": "scrape"
}' |
apify call datahamster/protected-page-fetch-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datahamster/protected-page-fetch-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wCojg1J69VAZUlgaf/builds/nEenRj0P1ZBrfW7ke/openapi.json