Pricing

from $500.00 / 1,000 page fetches

Web Scraper API - Anti-Bot Profile Selection, BYO Proxy

Best-effort page fetcher that runs a quick anti-bot recon, auto-selects the matching engine request profile, and fetches the page through your own proxy. Returns HTML length, detected stack, and latency.

Pricing

from $500.00 / 1,000 page fetches

Rating

0.0

(0)

Developer

Fenlo AI

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Web Scraper API — Auto Anti-Bot Profile Selection (Bring Your Own Proxy)

A best-effort page fetcher that first runs a quick anti-bot recon, then auto-selects the detection engine's matching request profile, and fetches the page through your proxy.

There is no promise of access. If a site blocks the request, that blocked response is what you get back — this Actor makes a best-effort fetch and reports the outcome; it does not claim to get past any defense.

Input

Field	Type	Default	Description
`url`	string	—	The URL to fetch.
`proxy`	string	—	Your own proxy URL, e.g. `http://user:pass@host:port`. Optional.
`extract_text`	boolean	`false`	Also return best-effort plain text extracted from the HTML.

Output

{
  "html": "<!doctype html><html>...</html>",
  "html_length": 13407,
  "stack": "cloudflare",
  "latency_ms": 842,
  "text": "..."
}

html is the full fetched page markup. text (best-effort plain text) is present only when extract_text is true.

Bring your own proxy

This Actor exposes a single proxy parameter by design. It has no proxy pool and no rotation — supply whichever proxy you already use.

How it works

A lightweight probe detects the site's anti-bot stack.
The detection engine's recommended request profile for that stack is selected automatically.
A single browser fetch is made through your proxy; the Actor returns the fetched page HTML, its length, the detected stack, and the round-trip latency (plus optional plain text).

For detection-only profiling across many URLs, see the companion recon-meta Actor.

Build & deploy

This Actor is self-contained: the Docker build context is this Actor folder. Apify confines the build context to the Actor's own directory, so a COPY cannot reach outside it.

This Actor depends on the repo-local fenlo_engine package (source at packages/engine — outside this folder). Because Docker cannot COPY across folders, the engine is vendored here as a pre-built pip wheel (fenlo_engine-*.whl). Build the wheel into this folder first, then build the image with this folder as the context:

# From the repo root: build the engine wheel into a throwaway dist/ and copy it
# into this Actor dir. Do NOT use `-o actors/stealth-scraper`: `uv build` writes a
# `.gitignore` containing `*` into its output dir, which would re-ignore this
# Actor's .actorignore and break `apify push`. dist/ is already git-ignored.
rm -f actors/stealth-scraper/fenlo_engine-*.whl
uv build --wheel packages/engine -o dist
cp dist/fenlo_engine-*.whl actors/stealth-scraper/

# From inside this Actor dir (context = ".")
cd actors/stealth-scraper
docker build -t fenlo-stealth-scraper .

The Dockerfile (referenced from .actor/actor.json) then:

installs the vendored engine with pip install fenlo_engine-*.whl, and
installs this Actor's requirements.txt (the Apify SDK),

so import fenlo_engine resolves inside the container. The base image is apify/actor-python-playwright, which ships the Chromium build this Actor uses for its browser fetch.

Trade-off: the wheel is a vendored copy, so rebuild it whenever packages/engine changes (otherwise the Actor ships a stale engine). The wheel is git-ignored; a .actorignore re-includes it so apify push still uploads it.

To publish on Apify: cd actors/stealth-scraper && apify push (rebuild the wheel first). See docs/launch/PUBLISH.md.

Anti-Bot Detection API - Batch Website Bot-Protection Profiler

fenlo-services/recon-meta

Detect a site's anti-bot stack (Cloudflare, DataDome, Akamai, PerimeterX, Imperva) and return a structured, batch-friendly pre-flight defense fingerprint as JSON.

Fenlo AI

Best Web Scraper API

crawlkit/crawlkit-scrape-api

Scrape any website and get clean markdown, HTML, metadata and links. Powered by CrawlKit.sh - supports stealth mode for anti-bot protection.

Crawlkit

Firecrawl MCP

red.cars/firecrawl-mcp

AI agents that need web data without anti-bot headaches. 20 tools for API-based web scraping, crawl, search, and extract — no proxy rotation, no stealth needed.

AutomateLab

Batch Anti-Bot Detection API - Profile Many URLs in One Run

fenlo-services/batch-recon

Detect the anti-bot stack (Cloudflare, DataDome, Akamai, PerimeterX, Imperva) for many URLs at once and return one JSON object of per-URL defense fingerprints (each with a recommended_action) plus a separate errors list.

Fenlo AI

Amazon Data Extractor

rizvi_ahmed/amazon-data-extractor

All-in-one Amazon data extraction — search, products, reviews, stores, and Q&A. 10 domains, anti-bot stealth, auto-pagination, CAPTCHA retry, and residential proxy support. Ready to scale.

Rizvi Ahmed

Insta Comment Bot

cozy_inclusion/insta-comment-bot

Instagram comment bot — 800/hr with proxy support

jerry rodriguez

Universal Web Scraper API

fayoussef/universal-scraper-api

Scrape any URL through a global, anti-bot-resistant Scraper API. JS rendering, geo-targeting, residential proxies, CSS-selector extraction, and optional LLM processing.

youssef farhan

5.0

ShieldRoute Analyzer

andreahlert/shieldroute-api

Estimate anti-bot risk and choose the best scraping route (http, hybrid, browser).

André Ahlert

Smart Page Fetcher — HTML, Markdown & Text

shelvick/smart-page-fetcher

Fetch a batch of URLs and get the page as HTML, Markdown, or clean text. Tries plain HTTP first, renders JavaScript in a real browser when needed, and escalates to stealth + residential proxy for Cloudflare-protected, bot-defended pages, per URL. Pay only for the difficulty each URL needed.

Scott Helvick

Trustpilot Review Scraper

traptija/estrattore-trustpilot

Extract text reviews, user names, and ratings from any Trustpilot profile. Built-in anti-bot bypass, clean JSON/CSV output ready for competitor analysis.