# AI Visibility Checker: Can ChatGPT Read Your Website? (`elraphh/ai-visibility-checker`) Actor

Check if ChatGPT, Claude, Perplexity, Gemini and Bing crawlers can read your website: robots.txt per AI bot, llms.txt, JSON-LD, text without JavaScript, canonical, meta tags, sitemap. Score /100 with one-line fixes. Free.

- **URL**: https://apify.com/elraphh/ai-visibility-checker.md
- **Developed by:** [Howseen AI](https://apify.com/elraphh) (community)
- **Categories:** SEO tools, AI
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Visibility Checker: check if ChatGPT and AI crawlers can read your website

**Check if ChatGPT, Claude, Perplexity, Gemini and Bing can read your website**, in seconds. Paste one or more URLs and get a score out of 100, the verdict for each AI crawler in your robots.txt, and one sentence per fix. Free, no AI calls, no login to any third party.

AI assistants can only recommend what they can read. A site can rank on Google and still be invisible to AI because its robots.txt blocks the AI search bots, because the content only appears after JavaScript runs, or because there is no structured data. This Actor finds those problems with plain HTTP requests.

### What it checks

| Check | Points | What it looks at |
|---|---|---|
| AI search and assistant crawlers in robots.txt | 25 | `OAI-SearchBot`, `ChatGPT-User`, `Claude-SearchBot`, `Claude-User`, `PerplexityBot`, `Perplexity-User`, `Googlebot`, `Bingbot`: allowed or blocked for this exact page, with the matching rule |
| AI training crawlers in robots.txt | 5 | `GPTBot`, `ClaudeBot`, `Google-Extended`, `Applebot-Extended` (blocking them is a valid choice, so it weighs little) |
| Indexable | 5 | No `noindex` in the meta robots tag or the `X-Robots-Tag` header |
| Text readable without JavaScript | 20 | Word count in the raw HTML, the way a crawler that does not run JavaScript sees it; flags pages rendered in the browser (React, Vue, Angular, Next.js shells) |
| Structured data (JSON-LD) | 15 | JSON-LD blocks, parse errors, schema.org types found (Organization, Product, SoftwareApplication, FAQPage, Article...) |
| Title tag | 5 | Present, 10 to 70 characters |
| Meta description | 5 | Present, 50 to 170 characters |
| Canonical | 5 | Points to the page itself, to another URL, or missing |
| Language | 3 | `<html lang>` declared |
| XML sitemap | 7 | Declared in robots.txt or found at `/sitemap.xml`, number of `<loc>` entries |
| llms.txt | 5 | `/llms.txt` exists as a text file (not an HTML fallback), its size, plus `/llms-full.txt` |

robots.txt is parsed like the standard says ([RFC 9309](https://www.rfc-editor.org/rfc/rfc9309)): the most specific user-agent group wins, then the longest matching rule, `Allow` wins ties, `*` and `$` wildcards are supported. robots.txt, llms.txt and the sitemap are fetched once per domain, even if you check many pages of the same site.

Grades: A = 90 and up, B = 75 and up, C = 60 and up, D = 40 and up, F below 40.

### How to use

1. Click **Try for free**.
2. Paste your URLs in **Website URLs to check**, one per line (home page, pricing page, a product page, a blog post). Up to 100 per run.
3. Click **Start**. Each URL takes a few seconds.
4. Open the **Output** tab: the **Overview** view shows the score and fixes, **All checks** shows every detail. The `SUMMARY` record in the key-value store gives the average score and the top 3 fixes per URL.

You can also call it from the API, a schedule (to watch your robots.txt after each deploy), Make, Zapier, n8n or any MCP client.

**Input example**

```json
{
  "urls": ["https://example.com", "https://example.com/pricing"],
  "timeoutSecs": 15
}
```

### Output example

Real result for `https://howseen.ai` (checks list shortened to the first 4 of 11, bots list shortened to 2 of 12):

```json
{
  "url": "https://howseen.ai",
  "finalUrl": "https://howseen.ai/",
  "statusCode": 200,
  "score": 100,
  "grade": "A",
  "checks": [
    {
      "id": "robots_ai_search",
      "label": "AI search and assistant crawlers allowed in robots.txt",
      "status": "pass",
      "points": 25,
      "maxPoints": 25,
      "detail": "All 8 search and assistant crawlers are allowed.",
      "recommendation": null
    },
    {
      "id": "robots_ai_training",
      "label": "AI training crawlers allowed in robots.txt",
      "status": "pass",
      "points": 5,
      "maxPoints": 5,
      "detail": "All training crawlers are allowed.",
      "recommendation": null
    },
    {
      "id": "indexable",
      "label": "Page is indexable (no noindex)",
      "status": "pass",
      "points": 5,
      "maxPoints": 5,
      "detail": "No noindex directive found.",
      "recommendation": null
    },
    {
      "id": "server_text",
      "label": "Text readable without JavaScript",
      "status": "pass",
      "points": 20,
      "maxPoints": 20,
      "detail": "2976 words in the raw HTML.",
      "recommendation": null
    }
  ],
  "recommendations": [],
  "robotsTxt": {
    "url": "https://howseen.ai/robots.txt",
    "found": true,
    "bots": {
      "OAI-SearchBot": {
        "company": "OpenAI",
        "purpose": "search",
        "allowed": true,
        "matchedGroup": "oai-searchbot",
        "rule": "Allow: /",
        "note": "ChatGPT search results"
      },
      "Googlebot": {
        "company": "Google",
        "purpose": "search",
        "allowed": true,
        "matchedGroup": "*",
        "rule": "Allow: /",
        "note": "Google Search, AI Overviews and AI Mode"
      }
    }
  },
  "llmsTxt": {
    "url": "https://howseen.ai/llms.txt",
    "found": true,
    "status": 200,
    "bytes": 10132,
    "llmsFullTxt": {
      "url": "https://howseen.ai/llms-full.txt",
      "found": true,
      "bytes": 5256
    }
  },
  "jsonLd": {
    "blocks": 5,
    "invalid": 0,
    "types": [
      "AggregateOffer",
      "Answer",
      "FAQPage",
      "Organization",
      "Person",
      "Question",
      "SoftwareApplication",
      "WebPage",
      "WebSite"
    ]
  },
  "content": {
    "wordCount": 2976,
    "likelyClientRendered": false,
    "frameworkHints": [],
    "htmlBytes": 298236
  },
  "canonical": {
    "href": "https://howseen.ai/",
    "pointsToSelf": true
  },
  "title": "Howseen AI: Track Your Brand in ChatGPT, Gemini & Perplexity",
  "lang": "en",
  "sitemap": {
    "found": true,
    "url": "https://howseen.ai/sitemap.xml",
    "source": "robots.txt",
    "urlCount": 4,
    "declaredInRobots": true
  },
  "howseen": "This checks if AI can read your site. To see if ChatGPT, Gemini, Perplexity and Google AI actually recommend your brand, try Howseen: https://howseen.ai/?utm_source=apify&utm_medium=actor&utm_campaign=ai-visibility-checker"
}
```

When something is wrong, `recommendations` lists one sentence per fix. Real recommendations returned for a JavaScript single-page app we tested (score 60/100, 1 word in the raw HTML):

- Most AI crawlers do not run JavaScript: render the main content on the server (SSR or static HTML) so they see it.
- Add JSON-LD (Organization plus Product, Service or FAQPage) in the server HTML so AI gets your facts in a machine-readable form.
- Add a /llms.txt that sums up who you are and links your key pages in Markdown (proposed standard, llmstxt.org).

### FAQ

**Does this tell me if ChatGPT recommends my brand?**
No. It checks whether AI crawlers are allowed in and whether they can read your page. Being readable is the first step; being recommended depends on what the AI finds about you across the web. To see if ChatGPT, Gemini, Perplexity and Google AI actually recommend your brand, try [Howseen](https://howseen.ai/?utm_source=apify\&utm_medium=actor\&utm_campaign=ai-visibility-checker).

**Which AI crawlers are checked, and where do the names come from?**
Only official user-agent tokens published by each company: [OpenAI](https://developers.openai.com/api/docs/bots) (GPTBot, OAI-SearchBot, ChatGPT-User), [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) (ClaudeBot, Claude-SearchBot, Claude-User), [Perplexity](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) (PerplexityBot, Perplexity-User), [Google](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) (Googlebot, Google-Extended), [Apple](https://support.apple.com/en-us/119829) (Applebot-Extended) and [Microsoft Bing](https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0) (Bingbot).

**Why does text without JavaScript matter?**
Vercel's crawler study found that "none of the major AI crawlers currently render JavaScript", with Googlebot and Applebot as exceptions ([source](https://vercel.com/blog/the-rise-of-the-ai-crawler)). If your content only appears after JavaScript runs, those crawlers see an empty page.

**Should I block GPTBot or ClaudeBot?**
It is your call. These bots collect data for model training; the search and assistant bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot...) are the ones that fetch pages to answer users. That is why training bots weigh 5 points and search bots 25.

**What is llms.txt?**
A proposed standard ([llmstxt.org](https://llmstxt.org/)): a Markdown file at the root of your site that sums up who you are and links your key pages for language models. It is optional and not used by every AI, so it weighs 5 points.

**Does it use a browser or AI?**
No. Plain HTTP requests only (no headless browser, no AI model), which keeps each run light and cheap on your Apify usage. It runs with 256 MB of memory.

**Is my site's firewall a problem?**
Some firewalls block automated requests. If a URL returns 403 or a challenge page, the result says so; AI crawlers may be blocked the same way, so check your firewall or CDN bot settings.

***

This checks if AI can read your site. To see if ChatGPT, Gemini, Perplexity and Google AI actually recommend your brand, try Howseen: https://howseen.ai/?utm_source=apify\&utm_medium=actor\&utm_campaign=ai-visibility-checker

# Actor input Schema

## `urls` (type: `array`):

Pages to check (home page, product page, blog post...). One URL per line, up to 100. "https://" is added if missing.

## `timeoutSecs` (type: `integer`):

How long to wait for each page, robots.txt, llms.txt and sitemap request.

## Actor input object example

```json
{
  "urls": [
    "https://apify.com"
  ],
  "timeoutSecs": 15
}
```

# Actor output Schema

## `results` (type: `string`):

One item per URL: score out of 100, every check with points, detail and a one-line recommendation, robots.txt verdict per AI crawler, llms.txt, JSON-LD types, word count without JavaScript.

## `summary` (type: `string`):

Average score and top 3 recommendations per URL.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("elraphh/ai-visibility-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://apify.com"] }

# Run the Actor and wait for it to finish
run = client.actor("elraphh/ai-visibility-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://apify.com"
  ]
}' |
apify call elraphh/ai-visibility-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,elraphh/ai-visibility-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/yYqblpSaUlecS3jS4/builds/eXxyY63RA5S6qLDB7/openapi.json