# Cloudflare Bypass (`pamberton/cloudflare-bypass`) Actor

Bypass Cloudflare protected routes. Works for API endpoints too unlike the web crawler.

- **URL**: https://apify.com/pamberton/cloudflare-bypass.md
- **Developed by:** [Pamberton](https://apify.com/pamberton) (community)
- **Categories:** Other, Automation, Developer tools
- **Stats:** 4 total users, 2 monthly users, 87.5% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $0.20 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Bypass Cloudflare - Advanced Web Scraper

**Bypass Cloudflare** is a premium Apify Actor designed to seamlessly crawl and extract data from websites protected by Cloudflare WAF (Web Application Firewall). By utilizing advanced browser emulation, fingerprint rotation, and routing traffic via the **Apify Proxy Network**, the Actor solves browser integrity challenges to retrieve raw JSON and HTML data without being blocked.

The easiest way to try the Actor is by keeping the prefilled **Start URL** configured to scrape Mongolian classified listings from the `unegui.mn` API. The Actor retrieves data directly from the target API and stores the merged list of classified items in your default Apify Dataset.

---

### 🚀 What Can This Actor Do?

Unlike standard web scrapers that get immediately blocked by Cloudflare's browser checks, this Actor introduces powerful anti-detection measures:

- 🕵️ **Fingerprint Rotation**: Rotates canvas, WebGL, WebRTC, audio, and font fingerprints across major operating systems including Windows, macOS, and Linux.
- 🌍 **Automatic GeoIP Spoofing**: Synchronizes browser locales, timezones, and geographic coordinates automatically with the egress country/location of your proxy server.
- ⚡ **Optimized Data Pipeline**: Supports blocking images to conserve bandwidth and speed up scraping time.
- 🔄 **Auto-Pagination and Merging**: Automatically crawls next-page links from JSON API payloads by matching specified pagination keys.
- 🤖 **Platform-Level Perks**: Enjoy built-in Apify platform benefits such as detailed run monitoring, API access, scheduling, webhook integrations, and proxy rotation.

---

### 📊 Extracted Datapoints

The Actor outputs structured data to the dataset. Below are the primary fields extracted during a scrape (example from `unegui.mn` listings):

| Field Name | Type | Description |
| :--- | :--- | :--- |
| **`id`** | `Number` | Unique identifier for the listing |
| **`title`** | `String` | Headline/Title of the classified advertisement |
| **`geometry`** | `Array` | Latitude and longitude coordinates where the item is located |
| **`absolute_url`** | `String` | Path component of the listing URL |
| **`price`** | `String` | Cost of the item, formatted in local currency (e.g. ₮) |
| **`image`** | `String` | URL of the primary listing thumbnail image |
| **`auction`** | `Boolean` | Flag indicating whether the listing is an auction |
| **`url`** | `String` | Canonical absolute URL to the listing |

---

### 📖 How to Use This Actor

Follow this quick guide to start scraping protected data:

1. **Configure the Start URLs**: Provide the target URL or API endpoint. For example:
   `https://www.unegui.mn/api/items/adverts-geometry/?rubric=1720...`
2. **Choose Fingerprint Spoofing**: Set your target operating system or leave it on **Random** to rotate fingerprints across runs.
3. **Configure Pagination & Merging Keys**: Set the JSON keys used by the target API for the next page URL (e.g., `next`) and the array containing item results (e.g., `results`).
4. **Set Page Limits**: Configure the maximum pages to limit pagination and avoid excessive compute unit consumption.
5. **Run the Actor**: Click the **Start** button on the Apify Console.
6. **Download Results**: Retrieve the dataset in your preferred format (JSON, CSV, Excel, XML).

---

### 🗃️ Input and Output Examples

#### Input Parameters

The Actor's configuration options are configured via the input tab. Below is a summary of the available fields:

```json
{
  "start_urls": [
    {
      "url": "https://www.unegui.mn/api/items/adverts-geometry/?rubric=1720&c=13575&ordering=&q=&cities=34&radius=158494&lat=47.961334664381894&lng=108.59620932333831&zoom_level=7.958010977973147"
    }
  ],
  "max_depth": 1,
  "headless": true,
  "os": "random",
  "block_images": false,
  "geoip": true,
  "pagination_key": "next",
  "merge_key": "results",
  "max_pages": 5,
  "navigation_timeout": 60
}
````

#### Output JSON Example

When successfully scraped, items are pushed to the default dataset. A single record looks like this:

```json
{
  "id": 10540220,
  "title": "Худ нисэх vitoria town 4 өрөө 89 мк",
  "geometry": [
    47.913249,
    106.938278
  ],
  "absolute_url": "/adv/10540220_khud-nisekh-vitoria-town-4-oroo-89-mk/",
  "price": "293,000,000 ₮",
  "image": "https://cdn1.unegui.mn/media/cache1/2a/d5/2ad593bb7e79924a1bc300d640aca55b.webp",
  "auction": false,
  "url": "https://www.unegui.mn/adv/10540220_khud-nisekh-vitoria-town-4-oroo-89-mk/"
}
```

***

### ⚖️ FAQ & Support

#### Troubleshooting

- **Actor gets blocked immediately**: Make sure you have a valid proxy configured. Cloudflare will flag and block most direct data center IPs without proxy rotation.
- **WAF challenges fail to resolve**: Try toggling headed mode (when running locally) or changing the spoofed OS fingerprint configuration.
- **Images fail to load or script errors**: Disabling images can sometimes break frontend scripts that wait for image load events. Set image blocking to `False` if the site layout fails to render.

For bugs, feedback, or custom development requests, please open an issue in the **Issues** tab on the Apify platform or contact the developer directly.

[Contact Us](mailto:contact@pamberton.com)

# Actor input Schema

## `start_urls` (type: `array`):

URLs to start with

## `max_depth` (type: `integer`):

Depth to which to scrape to

## `headless` (type: `boolean`):

Run browser in headless mode. Headless is faster, but headed can sometimes bypass stricter checks.

## `os` (type: `string`):

The operating system fingerprint to spoof. Selecting 'random' will rotate fingerprints.

## `block_images` (type: `boolean`):

Block image loading to save bandwidth and increase scraping speed.

## `geoip` (type: `boolean`):

Automatically spoof locale, timezone, and coordinates matching the proxy server IP.

## `pagination_key` (type: `string`):

The JSON key that contains the URL of the next page (e.g. 'next'). Set to empty string to disable pagination.

## `merge_key` (type: `string`):

The JSON key containing the array of results to merge across pages (e.g. 'results').

## `max_pages` (type: `integer`):

Maximum number of paginated pages to fetch for a given start URL.

## `navigation_timeout` (type: `integer`):

Timeout for page navigation in seconds.

## `proxyConfiguration` (type: `object`):

Select proxies to be used by the scraper.

## Actor input object example

```json
{
  "start_urls": [
    {
      "url": "https://pamberton.com"
    }
  ],
  "max_depth": 1,
  "headless": true,
  "os": "random",
  "block_images": false,
  "geoip": true,
  "pagination_key": "next",
  "merge_key": "results",
  "max_pages": 5,
  "navigation_timeout": 60,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "start_urls": [
        {
            "url": "https://pamberton.com"
        }
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("pamberton/cloudflare-bypass").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "start_urls": [{ "url": "https://pamberton.com" }],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("pamberton/cloudflare-bypass").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "start_urls": [
    {
      "url": "https://pamberton.com"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call pamberton/cloudflare-bypass --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=pamberton/cloudflare-bypass",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Cloudflare Bypass",
        "description": "Bypass Cloudflare protected routes. Works for API endpoints too unlike the web crawler.",
        "version": "0.0",
        "x-build-id": "VKhNuGz4wjJWKEJCX"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/pamberton~cloudflare-bypass/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-pamberton-cloudflare-bypass",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/pamberton~cloudflare-bypass/runs": {
            "post": {
                "operationId": "runs-sync-pamberton-cloudflare-bypass",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/pamberton~cloudflare-bypass/run-sync": {
            "post": {
                "operationId": "run-sync-pamberton-cloudflare-bypass",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "start_urls"
                ],
                "properties": {
                    "start_urls": {
                        "title": "Start URLs",
                        "type": "array",
                        "description": "URLs to start with",
                        "items": {
                            "type": "object",
                            "required": [
                                "url"
                            ],
                            "properties": {
                                "url": {
                                    "type": "string",
                                    "title": "URL of a web page",
                                    "format": "uri"
                                }
                            }
                        }
                    },
                    "max_depth": {
                        "title": "Maximum depth",
                        "type": "integer",
                        "description": "Depth to which to scrape to",
                        "default": 1
                    },
                    "headless": {
                        "title": "Headless Mode",
                        "type": "boolean",
                        "description": "Run browser in headless mode. Headless is faster, but headed can sometimes bypass stricter checks.",
                        "default": true
                    },
                    "os": {
                        "title": "Spoofed Operating System",
                        "enum": [
                            "random",
                            "windows",
                            "macos",
                            "linux"
                        ],
                        "type": "string",
                        "description": "The operating system fingerprint to spoof. Selecting 'random' will rotate fingerprints.",
                        "default": "random"
                    },
                    "block_images": {
                        "title": "Block Images",
                        "type": "boolean",
                        "description": "Block image loading to save bandwidth and increase scraping speed.",
                        "default": false
                    },
                    "geoip": {
                        "title": "Enable GeoIP Spoofing",
                        "type": "boolean",
                        "description": "Automatically spoof locale, timezone, and coordinates matching the proxy server IP.",
                        "default": true
                    },
                    "pagination_key": {
                        "title": "JSON Pagination Key",
                        "type": "string",
                        "description": "The JSON key that contains the URL of the next page (e.g. 'next'). Set to empty string to disable pagination.",
                        "default": "next"
                    },
                    "merge_key": {
                        "title": "JSON Merge Key",
                        "type": "string",
                        "description": "The JSON key containing the array of results to merge across pages (e.g. 'results').",
                        "default": "results"
                    },
                    "max_pages": {
                        "title": "Maximum Pages to Crawl",
                        "minimum": 1,
                        "type": "integer",
                        "description": "Maximum number of paginated pages to fetch for a given start URL.",
                        "default": 5
                    },
                    "navigation_timeout": {
                        "title": "Navigation Timeout",
                        "minimum": 1,
                        "type": "integer",
                        "description": "Timeout for page navigation in seconds.",
                        "default": 60
                    },
                    "proxyConfiguration": {
                        "title": "Proxy Configuration",
                        "type": "object",
                        "description": "Select proxies to be used by the scraper.",
                        "default": {
                            "useApifyProxy": true
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```