Bypass Cloudflare - Advanced Web Scraper

Bypass Cloudflare is a premium Apify Actor designed to seamlessly crawl and extract data from websites protected by Cloudflare WAF (Web Application Firewall). By utilizing advanced browser emulation, fingerprint rotation, and routing traffic via the Apify Proxy Network, the Actor solves browser integrity challenges to retrieve raw JSON and HTML data without being blocked.

The easiest way to try the Actor is by keeping the prefilled Start URL configured to scrape Mongolian classified listings from the unegui.mn API. The Actor retrieves data directly from the target API and stores the merged list of classified items in your default Apify Dataset.

---

🚀 What Can This Actor Do?

Unlike standard web scrapers that get immediately blocked by Cloudflare's browser checks, this Actor introduces powerful anti-detection measures:

• 🕵️ Fingerprint Rotation: Rotates canvas, WebGL, WebRTC, audio, and font fingerprints across major operating systems including Windows, macOS, and Linux.
• 🌍 Automatic GeoIP Spoofing: Synchronizes browser locales, timezones, and geographic coordinates automatically with the egress country/location of your proxy server.
• ⚡ Optimized Data Pipeline: Supports blocking images to conserve bandwidth and speed up scraping time.
• 🔄 Auto-Pagination and Merging: Automatically crawls next-page links from JSON API payloads by matching specified pagination keys.
• 🤖 Platform-Level Perks: Enjoy built-in Apify platform benefits such as detailed run monitoring, API access, scheduling, webhook integrations, and proxy rotation.

---

📊 Extracted Datapoints

The Actor outputs structured data to the dataset. Below are the primary fields extracted during a scrape (example from unegui.mn listings):

Field Name	Type	Description
id	Number	Unique identifier for the listing
title	String	Headline/Title of the classified advertisement
geometry	Array	Latitude and longitude coordinates where the item is located
absolute_url	String	Path component of the listing URL
price	String	Cost of the item, formatted in local currency (e.g. ₮)
image	String	URL of the primary listing thumbnail image
auction	Boolean	Flag indicating whether the listing is an auction
url	String	Canonical absolute URL to the listing

---

📖 How to Use This Actor

Follow this quick guide to start scraping protected data:

1. Configure the Start URLs: Provide the target URL or API endpoint. For example: https://www.unegui.mn/api/items/adverts-geometry/?rubric=1720...
2. Choose Fingerprint Spoofing: Set your target operating system or leave it on Random to rotate fingerprints across runs.
3. Configure Pagination & Merging Keys: Set the JSON keys used by the target API for the next page URL (e.g., next) and the array containing item results (e.g., results).
4. Set Page Limits: Configure the maximum pages to limit pagination and avoid excessive compute unit consumption.
5. Run the Actor: Click the Start button on the Apify Console.
6. Download Results: Retrieve the dataset in your preferred format (JSON, CSV, Excel, XML).

---

🗃️ Input and Output Examples

Input Parameters

The Actor's configuration options are configured via the input tab. Below is a summary of the available fields:

{
  "start_urls": [
    {
      "url": "https://www.unegui.mn/api/items/adverts-geometry/?rubric=1720&c=13575&ordering=&q=&cities=34&radius=158494&lat=47.961334664381894&lng=108.59620932333831&zoom_level=7.958010977973147"
    }
  ],
  "max_depth": 1,
  "headless": true,
  "os": "random",
  "block_images": false,
  "geoip": true,
  "pagination_key": "next",
  "merge_key": "results",
  "max_pages": 5,
  "navigation_timeout": 60
}

Output JSON Example

When successfully scraped, items are pushed to the default dataset. A single record looks like this:

{
  "id": 10540220,
  "title": "Худ нисэх vitoria town 4 өрөө 89 мк",
  "geometry": [
    47.913249,
    106.938278
  ],
  "absolute_url": "/adv/10540220_khud-nisekh-vitoria-town-4-oroo-89-mk/",
  "price": "293,000,000 ₮",
  "image": "https://cdn1.unegui.mn/media/cache1/2a/d5/2ad593bb7e79924a1bc300d640aca55b.webp",
  "auction": false,
  "url": "https://www.unegui.mn/adv/10540220_khud-nisekh-vitoria-town-4-oroo-89-mk/"
}

---

⚖️ FAQ & Support

Troubleshooting

• Actor gets blocked immediately: Make sure you have a valid proxy configured. Cloudflare will flag and block most direct data center IPs without proxy rotation.
• WAF challenges fail to resolve: Try toggling headed mode (when running locally) or changing the spoofed OS fingerprint configuration.
• Images fail to load or script errors: Disabling images can sometimes break frontend scripts that wait for image load events. Set image blocking to False if the site layout fails to render.

For bugs, feedback, or custom development requests, please open an issue in the Issues tab on the Apify platform or contact the developer directly.

Contact Us