# Bot Protection & WAF Detector | Public Signals (`keystonelabs/bot-protection-detector`) Actor

Identify exposed WAF, CAPTCHA and bot-protection technology signals on supplied websites. Export public observations for technical inventory. Does not bypass challenges, test defenses or establish security coverage.

- **URL**: https://apify.com/keystonelabs/bot-protection-detector.md
- **Developed by:** [Nick McNemar](https://apify.com/keystonelabs) (community)
- **Categories:** Developer tools, SEO tools, Lead generation
- **Stats:** 1 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 website checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### Inspect visible protection signals on websites

Bot Protection & WAF Detector: Identify exposed WAF, CAPTCHA and bot-protection technology signals on supplied websites. Export public observations for technical inventory. Does not bypass challenges, test defenses or establish security coverage.

### Start with a small example

1. Open Input, switch to JSON and paste the example below. Replace the example startUrls with your own research targets when ready.
2. Check the live Pricing tab and set a run spending limit.
3. Run the Actor, then open the **Workflow** output view. Inspect `url`, `security`, `cdn`, `matchesFilter`, `statusCode`, `error`.
4. Export JSON for nested data or CSV for a spreadsheet. Review a small sample before increasing the scope.

```json
{
  "startUrls": [
    "https://www.zendesk.com",
    "https://www.hp.com",
    "https://www.patagonia.com"
  ],
  "filterTechnologies": [
    "Cloudflare Bot Management",
    "Akamai Bot Manager",
    "DataDome",
    "PerimeterX",
    "HUMAN",
    "Imperva",
    "Kasada",
    "Shape Security",
    "F5 BIG-IP",
    "Cloudflare Turnstile",
    "reCAPTCHA",
    "hCaptcha",
    "Arkose Labs",
    "Sucuri",
    "Wordfence",
    "AWS WAF",
    "Azure Front Door",
    "Radware",
    "Fastly Next-Gen WAF",
    "Signal Sciences",
    "Queue-it",
    "Cloudflare"
  ],
  "onlyMatching": false
}
```

At the rate checked September 9, 2026, 3 saved rows cost $0.009 in Actor event fees ($3.00 per 1,000 rows). Actual returned rows determine the event fee. Taxes and other account services may be separate.

### What can I use it for?

- **Crawl and data-collection planning.** Before you schedule a large scrape, know which targets sit behind DataDome, Kasada or Akamai Bot Manager and which are open, and budget proxies and time accordingly.
- **Security sales.** Find sites with no bot protection at all, or on a competitor's product, and target them.
- **Security audits and due diligence.** Inventory which protection vendors a portfolio of client, subsidiary or acquisition-target sites actually runs.
- **Market research.** Measure the share of Cloudflare, Akamai, DataDome and PerimeterX across an industry or region.
- **Competitive intelligence.** See what your competitors have invested in at the edge.
- **Enriching your data.** Add `security` and `cdn` columns to a spreadsheet of domains in one run.

### What data does it return?

One record per website:

| Field | Meaning |
|---|---|
| `url`, `finalUrl`, `statusCode`, `title` | What you asked for, where it resolved to, and the page title |
| `security` | The protection services detected, e.g. `Cloudflare Bot Management`, `Akamai Bot Manager`, `DataDome`, `reCAPTCHA` |
| `matchesFilter` | `true` when the site uses any of the vendors you listed under **Vendors to look for** |
| `matchedTechnologies` | Which listed vendors were found |
| `cdn`, `reverseProxy` | The edge the protection runs on |
| `statusCode` | Worth watching: a `403` or `429` on a plain first request is itself evidence of an aggressive bot policy |
| `hosting`, `webServer` | Origin signals where visible |
| `technologyCount` | Total technologies detected |
| `technologies` | The complete list with categories, versions and confidence scores |
| `error` | `null` on success, otherwise why the site could not be checked |

Sites that time out, refuse the connection or answer with a challenge page are still returned — with the status code and whatever the challenge itself reveals — so input and output line up one to one.

#### Illustrative output

Values explain the output shape; they are not a live result or guaranteed field coverage.

```json
{
  "url": "https://www.zendesk.com",
  "finalUrl": "https://www.zendesk.com/",
  "statusCode": 200,
  "title": "Zendesk: The Complete Customer Service Solution",
  "security": "Cloudflare Bot Management",
  "cdn": "Cloudflare",
  "matchesFilter": true,
  "matchedTechnologies": ["Cloudflare Bot Management", "Cloudflare"],
  "technologyCount": 8,
  "technologies": [
    { "name": "Cloudflare Bot Management", "categories": ["Security"], "version": null, "confidence": 100, "website": "https://www.cloudflare.com/products/bot-management/" },
    { "name": "Cloudflare", "categories": ["CDN"], "version": null, "confidence": 100, "website": "https://www.cloudflare.com" }
  ],
  "error": null
}
```

### How to use it

1. Paste your domains under **Websites to check**, upload a file, or pass them through the API as `startUrls`. Bare domains like `example.com` are fine.
2. Leave **Vendors to look for** at its defaults for a broad inventory, or narrow it to one vendor to build a targeted list.
3. Switch on **Store only matching websites** to pay only for sites using the vendors you listed and discard the rest.
4. Click **Start** and download the results as JSON, CSV or Excel from the **Output** tab, or read them from the dataset via the API.

#### Input example

```json
{
  "startUrls": [
    "https://www.zendesk.com",
    "hp.com",
    "https://www.patagonia.com"
  ],
  "filterTechnologies": ["Cloudflare Bot Management", "Akamai Bot Manager", "DataDome", "PerimeterX"],
  "onlyMatching": false
}
```

Raise **Max concurrency** for large lists. Turn on **Proxy configuration** only if you want to see how a site treats a different IP class.

### How detection works

Protection vendors leave traces in the first response even when they let a request through: Cloudflare Bot Management's `__cf_bm` cookie and `cf-mitigated` header, Akamai's `_abck` and `bm_sz` cookies, DataDome's `datadome` cookie and `x-datadome` header, PerimeterX's `_px` cookies and `px-cdn` scripts, Kasada's `x-kpsdk-*` headers, Imperva's `visid_incap_*` and `incap_ses_*` cookies, the reCAPTCHA, hCaptcha and Turnstile script tags, and so on. The detector fetches each homepage once and checks headers, cookies, HTML, script sources and meta tags against more than 7,600 technology fingerprints, then reports every match with a confidence score.

The actor makes one ordinary request and reads the answer. It never attempts to solve, bypass or probe a challenge — a challenge page is simply reported as such, with the vendor it identifies.

The fingerprint database is the open-source webappanalyzer technology dataset, used under the GNU GPL v3. The matching engine in this actor is original code.

### Pricing

You pay per website saved to the dataset. With **Store only matching websites** on, that means per matching site; with it off, per site checked. There are no subscriptions, seats or minimums, and you can cap the maximum spend of a run before it starts.

### Scope and responsible use

The actor requests only the publicly served homepage of each URL you provide, exactly as a browser would, and reports what that single response reveals. It requires no login, no cookies and no credentials, performs no scanning, fuzzing or repeated probing, does not crawl beyond the page you give it, and does not attempt to solve or bypass any challenge or access control. You are responsible for using the results in accordance with the laws that apply to you and the terms of the websites you check.

### Limitations

- Protection that activates only on suspicious behaviour (rate, navigation patterns, JavaScript challenges) may not show on a single polite request; absence of a signal is not proof of absence of protection.
- Vendors that strip their identifying headers and cookies are reported only when other signals remain.
- Very large retail and media homepages that deliberately stall automated traffic can exceed the request timeout; they come back as rows with an `error` rather than being dropped.
- Only the URL you supply is fetched; login, checkout and API endpoints are often protected differently from the homepage.

### Something not detected?

Open an issue on this actor's **Issues** tab with the URL and the vendor you expected. Include a reproducible input and the expected signal so the report can be investigated.

### Interpret empty results and errors

A null technology field means no recognized signal was found in the inspected response; it does not prove absence. A returned error means that URL was not successfully analyzed. With `onlyMatching: true`, non-matches and errors are omitted from the dataset, so a zero-row result cannot distinguish an unavailable site from no matches. Start with `onlyMatching: false` when checking coverage.

Every saved row, including an error row, incurs the configured per-row event fee. Proxy selection and run duration can affect operating costs. Do not increase concurrency or enable a more expensive proxy merely to work around an unclear result.

For a reproducible problem, open this Actor’s Issues tab with a small public input, expected behavior and relevant error text. Remove tokens and confidential information. No response-time or uptime guarantee is offered.

### Use the result in an automation

In Make or n8n, use the Apify integration to run this Actor with the same JSON input, wait for completion, then retrieve its default dataset. Route failed runs and error rows to a review step before sending valid results to your spreadsheet or CRM. Scheduling does not make these Actors emit only new records: deduplicate downstream using the source URL or record ID.

For Node.js, install the official `apify-client` package and set your own `APIFY_TOKEN` environment variable. The following example starts a paid run with a small spending limit.

```javascript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const input = {
  "startUrls": [
    "https://www.zendesk.com",
    "https://www.hp.com",
    "https://www.patagonia.com"
  ],
  "filterTechnologies": [
    "Cloudflare Bot Management",
    "Akamai Bot Manager",
    "DataDome",
    "PerimeterX",
    "HUMAN",
    "Imperva",
    "Kasada",
    "Shape Security",
    "F5 BIG-IP",
    "Cloudflare Turnstile",
    "reCAPTCHA",
    "hCaptcha",
    "Arkose Labs",
    "Sucuri",
    "Wordfence",
    "AWS WAF",
    "Azure Front Door",
    "Radware",
    "Fastly Next-Gen WAF",
    "Signal Sciences",
    "Queue-it",
    "Cloudflare"
  ],
  "onlyMatching": false
};
const run = await client.actor('keystonelabs/bot-protection-detector').call(input, {
  memory: 512, timeout: 180, maxTotalChargeUsd: 0.10
});
if (run.status !== 'SUCCEEDED') throw new Error('Run did not succeed: ' + run.id);
const { items } = await client.dataset(run.defaultDatasetId).listItems({ limit: 100 });
console.log(items);
```

The first 100 rows are retrieved in this example. Use the dataset API pagination for a larger result. Keep credentials out of shared inputs and source files.

### A useful next step

- [Website Tech Stack Detector | Bulk URL Lookup](https://apify.com/keystonelabs/tech-stack-detector): qualify a website list by its technology. This is a separate Actor with its own input and price.

### Understand the run at a glance

Open **Run summary** in Output for counts of unique URLs, analyzed responses, failures, matches and saved rows. It remains useful when `onlyMatching` produces an empty dataset. The summary is written at normal completion and does not add a billable dataset row. Failed or aborted runs may not have a final summary; check run status first. The summary contains counts only, not a second copy of scraped content.

# Actor input Schema

## `startUrls` (type: `array`):

Domains or URLs to inspect. Each one becomes a result naming the protection services found, with a matchesFilter flag for the vendors you care about.

## `filterTechnologies` (type: `array`):

Bot-management, WAF, CAPTCHA and anti-fraud services that count as a match. The defaults cover the major vendors; narrow the list to find, for example, every site behind DataDome.

## `onlyMatching` (type: `boolean`):

Save only detected matches. Non-matches and error rows are omitted, so start with this off when reviewing coverage. Requires filterTechnologies.

## `maxConcurrency` (type: `integer`):

How many websites to analyze in parallel. Lower it if you are analyzing many pages on the same domain.

## `timeoutSecs` (type: `integer`):

Give up on a website after this many seconds. Large media and retail homepages can take 30 seconds or more to respond under load.

## `proxyConfiguration` (type: `object`):

Optional. Route requests through Apify Proxy if some target sites block your IP.

## Actor input object example

```json
{
  "startUrls": [
    "https://www.zendesk.com",
    "https://www.hp.com",
    "https://www.patagonia.com"
  ],
  "filterTechnologies": [
    "Cloudflare Bot Management",
    "Akamai Bot Manager",
    "DataDome",
    "PerimeterX",
    "HUMAN",
    "Imperva",
    "Kasada",
    "Shape Security",
    "F5 BIG-IP",
    "Cloudflare Turnstile",
    "reCAPTCHA",
    "hCaptcha",
    "Arkose Labs",
    "Sucuri",
    "Wordfence",
    "AWS WAF",
    "Azure Front Door",
    "Radware",
    "Fastly Next-Gen WAF",
    "Signal Sciences",
    "Queue-it",
    "Cloudflare"
  ],
  "onlyMatching": false,
  "maxConcurrency": 20,
  "timeoutSecs": 45,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset of analyzed websites. Each item lists detected technologies with category, version and confidence.

## `workflowJson` (type: `string`):

Selected columns for this workflow. Inspect errors and missing values before using the data.

## `workflowCsv` (type: `string`):

Download the workflow columns as CSV. Use JSON for full nested arrays.

## `summary` (type: `string`):

Counts of analyzed, failed, matched, saved and omitted URLs. Use this to interpret a zero-row result.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://www.zendesk.com",
        "https://www.hp.com",
        "https://www.patagonia.com"
    ],
    "filterTechnologies": [
        "Cloudflare Bot Management",
        "Akamai Bot Manager",
        "DataDome",
        "PerimeterX",
        "HUMAN",
        "Imperva",
        "Kasada",
        "Shape Security",
        "F5 BIG-IP",
        "Cloudflare Turnstile",
        "reCAPTCHA",
        "hCaptcha",
        "Arkose Labs",
        "Sucuri",
        "Wordfence",
        "AWS WAF",
        "Azure Front Door",
        "Radware",
        "Fastly Next-Gen WAF",
        "Signal Sciences",
        "Queue-it",
        "Cloudflare"
    ],
    "onlyMatching": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("keystonelabs/bot-protection-detector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        "https://www.zendesk.com",
        "https://www.hp.com",
        "https://www.patagonia.com",
    ],
    "filterTechnologies": [
        "Cloudflare Bot Management",
        "Akamai Bot Manager",
        "DataDome",
        "PerimeterX",
        "HUMAN",
        "Imperva",
        "Kasada",
        "Shape Security",
        "F5 BIG-IP",
        "Cloudflare Turnstile",
        "reCAPTCHA",
        "hCaptcha",
        "Arkose Labs",
        "Sucuri",
        "Wordfence",
        "AWS WAF",
        "Azure Front Door",
        "Radware",
        "Fastly Next-Gen WAF",
        "Signal Sciences",
        "Queue-it",
        "Cloudflare",
    ],
    "onlyMatching": False,
}

# Run the Actor and wait for it to finish
run = client.actor("keystonelabs/bot-protection-detector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://www.zendesk.com",
    "https://www.hp.com",
    "https://www.patagonia.com"
  ],
  "filterTechnologies": [
    "Cloudflare Bot Management",
    "Akamai Bot Manager",
    "DataDome",
    "PerimeterX",
    "HUMAN",
    "Imperva",
    "Kasada",
    "Shape Security",
    "F5 BIG-IP",
    "Cloudflare Turnstile",
    "reCAPTCHA",
    "hCaptcha",
    "Arkose Labs",
    "Sucuri",
    "Wordfence",
    "AWS WAF",
    "Azure Front Door",
    "Radware",
    "Fastly Next-Gen WAF",
    "Signal Sciences",
    "Queue-it",
    "Cloudflare"
  ],
  "onlyMatching": false
}' |
apify call keystonelabs/bot-protection-detector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,keystonelabs/bot-protection-detector"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/nDnvG3L6y4X9nraSZ/builds/osCWBCluwgJT7Mq1m/openapi.json