# Drishti.Anantasutra: AI Search Visibility Audit (GEO) (`motivating_bracelet/drishti-anantasutra`) Actor

See if ChatGPT, Claude, Perplexity, Google AI Overviews and 25+ AI crawlers can access your site: RFC 9309 robots.txt, llms.txt and schema checks plus a 0-100 GEO score with fixes.

- **URL**: https://apify.com/motivating_bracelet/drishti-anantasutra.md
- **Developed by:** [Bhavy Duneja](https://apify.com/motivating_bracelet) (community)
- **Categories:** SEO tools, AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 url auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Drishti.Anantasutra: AI Search Visibility Audit (GEO)

*Drishti* (दृष्टि) is Sanskrit for "sight". This Actor shows you how your website looks to AI search engines and
AI agents: which AI crawlers your robots.txt lets in, whether your pages can be read without JavaScript, whether you
publish an `llms.txt`, and what to fix first. It is part of the **Anantasutra** family of tools.

You give it one website or a list of up to 1,000 URLs. For each one you get a transparent 0-100 score, a per-crawler
access matrix for 28 AI crawlers (ChatGPT, Claude, Perplexity, Google, Bing, Apple, Meta, Amazon, Mistral and
others), and a prioritised list of fixes with copy-paste snippets.

### Who it is for

- **Site owners and marketers** who want to know whether ChatGPT search, Perplexity or Google AI Overviews can read
  and cite their site.
- **SEO and GEO agencies** that audit many client domains and need a consistent, explainable score per domain.
- **AI agents and developers** that need a machine-readable answer to "may AI crawlers access this page?" (it runs
  well from Claude, ChatGPT or Cursor through the Apify MCP server, see below).

### What it checks

| Area | What is checked | Points |
|---|---|---|
| AI crawler access | robots.txt verdict at `/` for 10 AI search crawlers and 7 user-triggered agents (RFC 9309 matching, longest rule wins) | 30 |
| robots.txt health | reachable, well formed, no rules outside groups | 5 |
| Indexability and snippets | HTTP status, `noindex`, `nosnippet`, canonical, redirects, response time | 20 |
| Content without JavaScript | main text present in the raw HTML, one H1, text-to-HTML ratio | 15 |
| Metadata and structured data | title, meta description, JSON-LD, OpenGraph, `lang` | 15 |
| LLM-specific files | `/llms.txt` present and well formed, `/llms-full.txt`, Markdown alternates | 10 |
| Sitemap | declared in robots.txt or found at `/sitemap.xml`, has URLs | 5 |

Blocking AI **training** crawlers (GPTBot, ClaudeBot, CCBot ...) is a business decision, so it is reported but never
lowers the score. Checks that could not run (for example because your robots.txt disallows this auditor) are
skipped and excluded from the score; `scoreCoverage` tells you how many of the 100 points were actually evaluated.

### Input

```json
{
  "urls": ["https://example.com", "example.org/pricing"],
  "pathsToCheck": ["/blog/", "/docs/"],
  "checkLlmsFullTxt": true,
  "includeBotDetails": true,
  "maxConcurrency": 3,
  "requestTimeoutSecs": 15
}
```

- `urls`: domains or full page URLs. For a domain the home page is audited; robots.txt, llms.txt and the sitemap are
  always checked at the site root.
- `pathsToCheck`: extra paths evaluated against robots.txt for every crawler (not fetched).
- `requestTimeoutSecs`: raise it for slow sites.

### Output

One dataset row per input URL. Abridged example:

```json
{
  "url": "https://www.example.com/",
  "status": "ok",
  "score": 100,
  "grade": "A",
  "scoreCoverage": 100,
  "summary": "AI visibility score 100/100 (A). AI search crawlers allowed: 10/10; user-triggered agents allowed: 7/7; training crawlers blocked: 2/10. llms.txt: valid. Main content without JavaScript: ok.",
  "searchBotsAllowed": "10/10",
  "userBotsAllowed": "7/7",
  "trainingBotsBlocked": "2/10",
  "llmsTxtStatus": "valid",
  "topRecommendation": "No critical issues found",
  "blockedAiCrawlers": ["GPTBot", "CCBot"]
}
```

Every row also contains `recommendations` (priority, area, title, detail, fix snippet), `scoreBreakdown` (every
check with points and the reason), `aiCrawlers` (per crawler: matched robots.txt group, deciding rule, link to the
operator's documentation), `robotsTxt`, `llmsTxt`, `llmsFullTxt`, `sitemap` and `page`. The **Recommendations** view
of the dataset lists one fix per row, highest priority first.

#### Results without a score

A failure to reach a site is not a verdict on its AI visibility, so these rows have `score: null` and
`grade: null` instead of a misleading 0/100 "F":

| Situation | `status` | Charged? |
|---|---|---|
| No answer at all: timeout (retried once), DNS error, invalid HTTPS certificate, connection refused | `error` | No |
| The address is not a public website (private network, localhost, cloud metadata) | `error` | No |
| The URL is a PDF, image, JSON API or other non-HTML file | `error` | No |
| Invalid input, for example `12345`, an e-mail address such as `info@example.com`, two addresses in one entry, or an object without a `url` field | `error` | No |
| The same address appears again in the input (`example.com`, `EXAMPLE.com/`, `example.com#top` ...) | `duplicate` | No |
| robots.txt answers HTTP 5xx or 429, or the page cannot be fetched or answers this audit with HTTP 403 (bot protection), 429 (throttling) or 5xx | `ok` | Yes |

In the last case the site did answer, so you still get the robots.txt result, the AI crawler matrix, llms.txt and
sitemap checks. Under RFC 9309 a robots.txt that answers 5xx or 429 means "disallow everything" for compliant
crawlers while it lasts, so re-run later and fix /robots.txt if it persists. A page that answers this audit with 403,
429 or 5xx is not graded either: AI crawlers may get a different answer, so check your firewall or bot-protection
rules and re-run instead of reading it as a failing page. A page that does not exist (404, 410) is still scored.

### Pricing

Pay per event, no subscription. You only pay for audits of sites that answered.

| Event | Price | When |
|---|---|---|
| URL audited | $0.004 | Once per unique address whose site answered (see "Results without a score" for what is never charged) |
| Actor start | $0.00005 | Charged automatically by Apify at the start of each run (per GB of memory) |

Auditing 1,000 domains costs about $4. Set "Maximum cost per run" in the run options to cap spending: the Actor
audits as many URLs as fit and tells you how many were skipped. Duplicates, unreachable sites, non-HTML URLs and
invalid inputs are never charged.

### Use it from Claude, ChatGPT or Cursor (MCP)

The [Apify MCP server](https://mcp.apify.com) exposes this Actor as a tool to AI assistants.

- **Claude Desktop / Claude Code, Cursor and other MCP clients:** add the hosted server
  `https://mcp.apify.com/?actors=motivating_bracelet/drishti-anantasutra` and sign in with your Apify account, or run it
  locally with `npx -y @apify/actors-mcp-server --actors motivating_bracelet/drishti-anantasutra` and your Apify API token
  in the `APIFY_TOKEN` environment variable.
- **ChatGPT:** add `https://mcp.apify.com` as a custom connector (MCP) where your plan allows it.

Then ask, for example: "Audit example.com with Drishti and list the three most important fixes." The assistant
passes `urls` as input and reads the `summary`, `topRecommendation` and `recommendations` fields.

### Limits

- Up to 1,000 URLs per run, 10 extra robots.txt paths, 10 sites in parallel; requests to one site are sequential.
- Raw HTML only: the page is not rendered in a browser, which is how most AI crawlers read it too.
- Page downloads are capped at 3 MB, robots.txt at 500 KiB (as RFC 9309 allows), `llms-full.txt` at 64 KB.
- Only public websites over http or https. Private, loopback, link-local and other reserved addresses are refused,
  including through DNS names and redirects.
- The auditor identifies itself honestly in its User-Agent (`AIVisibilityAudit`) and obeys your robots.txt for
  itself. If your firewall or bot protection drops or refuses unknown bots (timeout, HTTP 403 or 429), the result says
  it could not be checked instead of giving a grade.
- robots.txt is a policy, not a lock: the matrix shows what compliant crawlers may do. The crawler catalog was last
  checked against operator documentation on the date in `botsCatalogCheckedAt`.

### FAQ

**Why is my score null instead of a number?** The site or page did not answer (timeout, DNS, certificate error),
robots.txt answered with a server error, the page answered this audit with a block (403), throttling (429) or a
server error (5xx), or the URL is not an HTML page. The `summary` and `topRecommendation` say
which. Rows without an HTTP answer are not charged.

**Does blocking GPTBot hurt my score?** No. Training crawlers are reported for information only. Blocking AI
*search* crawlers such as OAI-SearchBot or PerplexityBot does lower the score, because it removes you from AI answers.

**I entered the same domain several times. Am I charged several times?** No. Entries that normalise to the same
address are audited and charged once; the repeats get a free `duplicate` row pointing to the first one. Note that
`http://` and `https://`, or `www.` and bare-domain forms (`www.example.com` and `example.com`), are different sites
with their own robots.txt, so each form you enter is audited and charged. The run log lists such pairs before the
audit starts, so keep only the forms you need.

**Can I audit a PDF or an API endpoint?** The Actor tells you it is not a web page and does not charge for it. Audit
the HTML page that links to it instead.

**Does it log in, submit forms or bypass bot protection?** No. It reads public pages the way a polite crawler does.

**How do I fix the issues?** Each recommendation has a `fix` field with a snippet (robots.txt lines, meta tags, an
llms.txt template). To generate an llms.txt for your site, see the sibling Actor **Suchi.Anantasutra**.

# Actor input Schema

## `urls` (type: `array`):

Domains (example.com) or full URLs (https://example.com/pricing). For a domain the homepage is audited; for a full URL that page is audited. robots.txt, llms.txt and the sitemap are always checked at the site root. http:// and https://, or www. and bare-domain forms of one address, are separate sites with their own robots.txt: each form you enter is audited and billed, and the run log lists such pairs before auditing.

## `pathsToCheck` (type: `array`):

Paths evaluated for every AI crawler in addition to / and the audited page, for example /blog/ or /docs/. Only robots.txt rules are evaluated for these paths; they are not fetched.

## `checkLlmsFullTxt` (type: `boolean`):

Also check whether /llms-full.txt exists (only the first 64 KB are downloaded).

## `includeBotDetails` (type: `boolean`):

Include the full per-crawler matrix (matched robots.txt group, deciding rule, operator documentation link). Turn off for smaller results.

## `maxConcurrency` (type: `integer`):

How many sites are audited at the same time. Requests to a single site are always sequential.

## `requestTimeoutSecs` (type: `integer`):

Timeout for each HTTP request.

## Actor input object example

```json
{
  "urls": [
    "https://apify.com",
    "https://crawlee.dev/docs/quick-start"
  ],
  "pathsToCheck": [
    "/"
  ],
  "checkLlmsFullTxt": true,
  "includeBotDetails": true,
  "maxConcurrency": 3,
  "requestTimeoutSecs": 15
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset items, one per audited URL.

## `recommendations` (type: `string`):

The same dataset with one row per recommendation.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("motivating_bracelet/drishti-anantasutra").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://apify.com"] }

# Run the Actor and wait for it to finish
run = client.actor("motivating_bracelet/drishti-anantasutra").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://apify.com"
  ]
}' |
apify call motivating_bracelet/drishti-anantasutra --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,motivating_bracelet/drishti-anantasutra"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iOr18Xq2wwe2BynAP/builds/HCh2beXj4k2NfZ90X/openapi.json