# AI Agent-Readiness Bulk Auditor (llms.txt, Markdown, AI Bots) (`growth-installations/agent-readiness-bulk-auditor`) Actor

Score any list of websites 0-100 on how well ChatGPT, Claude, Perplexity and Gemini agents can read them: robots.txt AI-bot blocks, firewall blocks, markdown negotiation, llms.txt, sitemap, JS-only pages, schema, hidden FAQs. Lead flag + pitch line per site.

- **URL**: https://apify.com/growth-installations/agent-readiness-bulk-auditor.md
- **Developed by:** [Growth Installations](https://apify.com/growth-installations) (community)
- **Categories:** SEO tools, Lead generation, AI
- **Security:** Limited permissions
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$50.00 / 1,000 domain auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Agent-Readiness Bulk Auditor: llms.txt, Markdown, AI Bots, Schema

**$0.05 per audited domain.** Unreachable domains are free.

Paste a list of websites and get a 0-100 score for how well ChatGPT, Claude, Perplexity and Gemini agents can read each one. Every row includes the fixes, a lead flag and a pitch line you can use in cold email.

### What does it do?

For every domain it runs about 8 light requests and checks:

| Check | Weight | What fails it |
|---|---|---|
| AI bots allowed in robots.txt | 20 | GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended and others disallowed |
| Firewall lets AI agents in | 10 | Homepage returns 403/429/challenge to an AI user agent but 200 to a browser |
| Markdown served | 15 | No markdown answer to `Accept: text/markdown` and no `.md` copy |
| /llms.txt | 10 | Missing or not markdown |
| Sitemap | 10 | No sitemap in robots.txt or at /sitemap.xml |
| Content in raw HTML | 15 | JS-only shell; agents that don't run JavaScript see an empty page |
| Schema | 10 | No Organization/LocalBusiness/Product/FAQ JSON-LD |
| FAQ readable in HTML | 5 | FAQ answers hidden behind JS accordions or missing FAQPage schema |
| Meta basics | 5 | Missing title, description or canonical |

### Who is it for?

AI SEO / GEO agencies and AI-automation agencies that want a fast list of prospects with an obvious, fixable problem, and in-house SEOs auditing a portfolio of sites.

### How much does it cost?

| Event | Price | What you get |
|---|---|---|
| Domain audited | $0.05 | One unique domain scored 0-100 with issues, fixes, lead flag and pitch line |

1,000 domains = $50. No third-party scrapers are used, so there are no extra upstream costs beyond normal Apify compute. Set a maximum charge per run in the Console to cap spend.

### Input example

```json
{
  "domains": ["stripe.com", "anthropic.com", "nectiv.com", "brightlocal.com"],
  "leadThreshold": 60,
  "agencyName": "Your Agency"
}
```

### Output

One dataset row per domain: `domain`, `score`, `grade`, `isLead`, `aiSearchBotsBlocked`, `aiBotsBlocked`, `aiUserAgentStatus`, `markdownServed`, `llmsTxt`, `sitemap`, `rawHtmlWords`, `jsShell`, `faqStatus`, `schemaTypes`, `emails`, `issueCount`, `issues`, `pitch`, `checks`.
Key-value store: `REPORT_MD`, `REPORT_HTML` (white-label, print to PDF), `REPORT_CSV`.

### Sample output (live run, Nov 3, 2026)

| domain | score | grade | isLead | top issue |
|---|---|---|---|---|
| stripe.com | 81 | B | false | No markdown served for `Accept: text/markdown` |
| brightlocal.com | 75 | B | false | robots.txt blocks Applebot-Extended, CCBot, meta-externalagent |
| gilmanheatingandcooling.com | 74 | B | false | No markdown version, no /llms.txt |
| ahrefs.com | 73 | B | false | No markdown version, no /llms.txt |
| anthropic.com | 63 | C | false | No markdown version, no /llms.txt, no schema |
| nectiv.com | 53 | D | **true** | No markdown, no /llms.txt, no sitemap, no schema |

Pitch line generated for nectiv.com: *"Ran nectiv.com through an AI-agent readiness check: 53/100 (grade D). Biggest gaps: Serve a markdown version when requested with 'Accept: text/markdown'..."*

### FAQ

**Does it crawl the whole site?** No. It checks the homepage plus the robots.txt, llms.txt, sitemap and markdown endpoints, so it is fast and cheap for big lists.

**Why is the score different with a proxy?** Firewalls treat datacenter IPs differently. Leave the proxy off for the most accurate AI-agent firewall test.

### Use it from n8n, Make, Claude or Cursor

- MCP: add `https://mcp.apify.com/?actors=growth-installations/agent-readiness-bulk-auditor` to Claude, Cursor or any MCP client.
- n8n or Make: add an HTTP Request step that POSTs your input JSON to the API endpoint below and returns the dataset items.
- API: `POST https://api.apify.com/v2/acts/growth-installations~agent-readiness-bulk-auditor/run-sync-get-dataset-items` with header `Authorization: Bearer <your Apify API token>`

# Actor input Schema

## `domains` (type: `array`):

Domains or URLs to audit, one per line (up to 5,000). Duplicates are removed. Each unique domain is one 'domain-audit' charge.

## `leadThreshold` (type: `integer`):

Sites scoring under this are flagged isLead=true and get a cold-email pitch line. Filter the dataset on isLead to get your prospect list.

## `agencyName` (type: `string`):

Added to the pitch line and the HTML report header, so you can send the report white-labeled.

## `concurrency` (type: `integer`):

Higher is faster. Each site gets about 8 light requests (homepage, robots.txt, llms.txt, sitemap, markdown and AI-user-agent checks).

## `timeoutSecs` (type: `integer`):

Sites slower than this are reported as unreachable for that check.

## `useApifyProxy` (type: `boolean`):

Route requests through Apify datacenter proxy. Leave off unless many sites rate-limit you; the firewall test is most accurate without a proxy.

## Actor input object example

```json
{
  "domains": [
    "stripe.com",
    "anthropic.com",
    "nectiv.com"
  ],
  "leadThreshold": 60,
  "agencyName": "",
  "concurrency": 10,
  "timeoutSecs": 15,
  "useApifyProxy": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `report_md` (type: `string`):

No description

## `report_html` (type: `string`):

No description

## `report_csv` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "anthropic.com",
        "nectiv.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("growth-installations/agent-readiness-bulk-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "stripe.com",
        "anthropic.com",
        "nectiv.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("growth-installations/agent-readiness-bulk-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "anthropic.com",
    "nectiv.com"
  ]
}' |
apify call growth-installations/agent-readiness-bulk-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,growth-installations/agent-readiness-bulk-auditor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/atsBh6at824wfGfXq/builds/sAX0ZE70l2MZrI3G6/openapi.json