# Anti-Bot Detection API - Batch Website Bot-Protection Profiler (`fenlo-services/recon-meta`) Actor Detect a site's anti-bot stack (Cloudflare, DataDome, Akamai, PerimeterX, Imperva) and return a structured, batch-friendly pre-flight defense fingerprint as JSON. - **URL**: https://apify.com/fenlo-services/recon-meta.md - **Developed by:** [Fenlo AI](https://apify.com/fenlo-services) (community) - **Categories:** Agents, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Anti-Bot Detection API — Batch Website Bot-Protection Profiler Programmatic, batch-friendly **detection** of the anti-bot stack protecting a website. Point it at a URL and get back structured JSON describing the defense you are up against — a pre-flight fingerprint for your own scraper fleet. This is a **detection-as-intelligence** tool. It reports *what* defenses a site runs so you can plan with your own tooling. It does not provide instructions for getting past those defenses, and it makes no promise of access. ### What it detects - The stack: Cloudflare, DataDome, Akamai, PerimeterX, Imperva, or none - A confidence score and the raw signals that triggered the match - A descriptive `defense_profile` (for example, *"Cloudflare present; expect an interactive JS challenge"*) - A coarse `difficulty_estimate`: `easy` | `medium` | `hard` | `unlikely` - `js_challenge_likely`: whether an interactive JS challenge is expected - `recommended_profile`: the request profile the detection engine would select - `recommended_action`: an honest, machine-branchable hint for how to **approach the target with your own tooling** — `standard_request`, `browser_render_recommended`, or `high_defense_expect_friction`. It describes the kind of request to plan for; it never describes getting past a defense and is not a promise of access. ### Input | Field | Type | Default | Description | |-------|------|---------|-------------| | `url` | string | — | The URL to fingerprint. | | `deep_playwright` | boolean | `false` | If the cheap HTTP probe cannot identify the stack, render the page in a real browser and re-inspect. Slower, costs more. | ### Output ```json { "url": "https://shop.example.com", "stack": "cloudflare", "confidence": 0.95, "recommended_profile": "stealth", "recommended_action": "browser_render_recommended", "signals": ["header:cf-ray"], "defense_profile": "Cloudflare present; expect an interactive JS challenge (e.g. Turnstile) on protected routes.", "js_challenge_likely": true, "difficulty_estimate": "hard" } ```` ### How it works A lightweight HTTP probe inspects the response headers and the first few KB of the body for known signatures. Only when that probe cannot identify the stack **and** you set `deep_playwright: true` does the Actor escalate to a full browser render and re-inspect — keeping the common path cheap and fast. The batch JSON output is the point: pipe many URLs through it and let your pipeline decide what to do, instead of checking sites one at a time by hand. ### Bring your own proxy For an actual fetch (not just detection), pair this with the companion **stealth-scraper** Actor, which accepts a single proxy parameter. ### Pricing Pay-per-event. Lightweight recon events are inexpensive; a deep browser render costs more because it runs a real browser. ### Build & deploy **This Actor is self-contained: the Docker build context is this Actor folder.** Apify confines the build context to the Actor's own directory, so a `COPY` cannot reach outside it. This Actor depends on the repo-local `fenlo_engine` package (source at `packages/engine` — outside this folder). Because Docker cannot `COPY` across folders, the engine is vendored here as a pre-built pip wheel (`fenlo_engine-*.whl`). Build the wheel into this folder first, then build the image with this folder as the context: ```bash ## From the repo root: build the engine wheel into a throwaway dist/ and copy it ## into this Actor dir. Do NOT use `-o actors/recon-meta`: `uv build` writes a ## `.gitignore` containing `*` into its output dir, which would re-ignore this ## Actor's .actorignore and break `apify push`. dist/ is already git-ignored. rm -f actors/recon-meta/fenlo_engine-*.whl uv build --wheel packages/engine -o dist cp dist/fenlo_engine-*.whl actors/recon-meta/ ## From inside this Actor dir (context = ".") cd actors/recon-meta docker build -t fenlo-recon-meta . ``` The `Dockerfile` (referenced from `.actor/actor.json`) then: 1. installs the vendored engine with `pip install fenlo_engine-*.whl`, and 2. installs this Actor's `requirements.txt` (the Apify SDK), so `import fenlo_engine` resolves inside the container. The base image is `apify/actor-python-playwright`, which ships a Chromium matched to the engine's Playwright dependency (used by the optional `deep_playwright` path). **Trade-off:** the wheel is a vendored copy, so **rebuild it whenever `packages/engine` changes** (otherwise the Actor ships a stale engine). The wheel is git-ignored; a `.actorignore` re-includes it so `apify push` still uploads it. To publish on Apify: `cd actors/recon-meta && apify push` (rebuild the wheel first). See `docs/launch/PUBLISH.md`. # Actor input Schema ## `url` (type: `string`): The URL to fingerprint for anti-bot defenses. ## `deep_playwright` (type: `boolean`): If the lightweight HTTP probe cannot identify the stack, render the page in a real browser and re-inspect. Slower and costs more. ## Actor input object example ```json { "url": "https://www.g2.com", "deep_playwright": false } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "url": "https://www.g2.com" }; // Run the Actor and wait for it to finish const run = await client.actor("fenlo-services/recon-meta").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "url": "https://www.g2.com" } # Run the Actor and wait for it to finish run = client.actor("fenlo-services/recon-meta").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "url": "https://www.g2.com" }' | apify call fenlo-services/recon-meta --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=fenlo-services/recon-meta", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Anti-Bot Detection API - Batch Website Bot-Protection Profiler", "description": "Detect a site's anti-bot stack (Cloudflare, DataDome, Akamai, PerimeterX, Imperva) and return a structured, batch-friendly pre-flight defense fingerprint as JSON.", "version": "0.1", "x-build-id": "3hcOlWfpl8wLhcJHA" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/fenlo-services~recon-meta/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-fenlo-services-recon-meta", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/fenlo-services~recon-meta/runs": { "post": { "operationId": "runs-sync-fenlo-services-recon-meta", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/fenlo-services~recon-meta/run-sync": { "post": { "operationId": "run-sync-fenlo-services-recon-meta", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "url" ], "properties": { "url": { "title": "Target URL", "type": "string", "description": "The URL to fingerprint for anti-bot defenses." }, "deep_playwright": { "title": "Deep browser check", "type": "boolean", "description": "If the lightweight HTTP probe cannot identify the stack, render the page in a real browser and re-inspect. Slower and costs more.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```