# AI Visibility & Competitor Source Gaps (`nawfaltech/ai-visibility-competitor-source-gaps`) Actor

Find the pages that make AI recommend your competitors. Asks Perplexity, ChatGPT and Gemini about your category, collects the sources they cite, then fetches those pages to verify which ones mention competitors but not your brand.

- **URL**: https://apify.com/nawfaltech/ai-visibility-competitor-source-gaps.md
- **Developed by:** [Nawfalpro](https://apify.com/nawfaltech) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $200.00 / 1,000 ai visibility analyses

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Visibility & Competitor Source Gaps

**Find the pages that make AI recommend your competitors, and check whether those pages mention your brand.**

The Actor asks AI engines the questions your customers ask: Perplexity by default, with ChatGPT (web search) and Gemini available for a full scan. It collects the sources the answers cite, then fetches each of those pages.
The goal is to confirm, from the real page content, which pages mention your competitors but not you.

### What it does

1. Builds up to 10 questions from your category and location, using fixed templates (no AI is used to write them). You can also supply your own questions.
2. Sends the questions to the AI engines you select (Perplexity by default).
3. Measures how often your brand and your competitors are mentioned in the answers.
4. Collects every source URL the answers cite and classifies each one: owned, forum, review site, listicle, publication, directory or other.
5. Fetches the most promising cited pages and checks their text for your brand and your competitors.
6. Returns a prioritized list of **source gaps**, plus a CSV of verified opportunities you can use for outreach.

### Who it is for

- Local businesses and agencies checking how AI assistants recommend businesses in a city.
- SaaS and e-commerce marketers doing GEO / AEO (generative/answer engine optimization).
- PR and outreach teams looking for articles, lists and directories to get included in.

### Input

| Field | Required | Description |
|---|---|---|
| Brand | yes | Your business name |
| Website | yes | Your website, e.g. `https://example.com` |
| Category | yes | What you are, e.g. `restaurant`, `CRM software` |
| Country | yes | Your market |
| City | no | For local businesses |
| Competitors | no | Up to 5. Accepted formats: `Name`, `Name \| domain.com`, `Name (domain.com)` or `domain.com`. If you leave this empty, competitors are discovered automatically. |
| Custom questions | no | Up to 25. These replace the standard questions. |
| AI engines | no | Default: **Perplexity only** (quick scan, 10 searches). Select ChatGPT and Gemini too for the **full analysis** (30 searches). See [Cost](#cost). |
| Samples per question | no | 1–10 (default 1) |
| Pages to check | no | 0–100 (default 25) |

Default quick scan (Perplexity only):

```json
{
  "brand": "Example Restaurant",
  "website": "https://example.com",
  "category": "Restaurant",
  "country": "Saudi Arabia",
  "city": "Riyadh"
}
```

Optional full scan: add `"engines": ["perplexity", "chatgpt", "gemini"]`. That runs 30 searches instead of 10 (see [Cost](#cost)).

### Output

| Where | What |
|---|---|
| **Dataset** (table) | One row per source gap (page × competitor), sorted high → medium → unverified. If a run finds no gaps, it holds a single row with `recordType: "run_summary"` that explains why (not an opportunity) |
| Key-value store `OUTPUT` | The full JSON report: `run`, `brandVisibility`, `topCompetitors`, `responses`, `citations`, `pages`, `sourceGaps`, `uncheckedSources`, `summary`, `errors`, `notes` |
| Key-value store `OPPORTUNITIES.csv` | Verified opportunities only (high + medium). Columns: `url, domain, sourceType, competitor, brand, prompt, engine, verification, priority` |
| Key-value store `RAW_RESPONSES` | The full raw answers exactly as the engines returned them, for debugging |

Example source gap (default quick scan):

```json
{
  "url": "https://www.timeout.com/riyadh/restaurants/best-restaurants-in-riyadh",
  "domain": "timeout.com",
  "title": "The 15 best restaurants in Riyadh",
  "sourceType": "listicle_comparison",
  "competitor": "Najd Heritage Kitchen",
  "brand": "Example Restaurant",
  "verification": "verified_competitor_only",
  "priority": "high",
  "reason": "Page fetched: \"Najd Heritage Kitchen\" found 2× in main content; \"Example Restaurant\" not found anywhere on the page",
  "competitorMentionsOnPage": 2,
  "timesCited": 10,
  "prompt": "What are the best restaurant choices for customers in Saudi Arabia?",
  "engine": "perplexity",
  "engines": ["perplexity"]
}
```

Example visibility and top competitors (default quick scan):

```json
"brandVisibility": { "mentions": 3, "responses": 8, "rate": 0.375, "percentage": 37.5, "basis": "discovery_prompts",
  "byEngine": { "perplexity": { "mentions": 3, "responses": 8, "rate": 0.375, "percentage": 37.5 } } },
"topCompetitors": [
  { "rank": 1, "name": "Najd Heritage Kitchen", "mentionRate": 1, "mentions": 8, "totalResponses": 8, "origin": "discovered", "confidence": "medium" }
]
```

In a full scan, `engines` lists every engine that cited the page, and `byEngine` has one entry per engine.

(These examples come from the Actor's built-in test fixtures. The business names in them are fictional.)

### What "verified competitor-only" means

A page is marked `verified_competitor_only` **only when all of the following are true**:

- the page was actually downloaded (HTTP 200, HTML, readable text of at least 150 words);
- a competitor was found in the page's **main content** (navigation, header, footer and sidebars are ignored);
- your brand was **not** found **anywhere** on the page. That check covers the main text, navigation, footer, title, meta tags and image alt text, plus any link to your domain.

The check is deliberately lopsided: competitors must be in the main content, while your brand is searched everywhere. As a result, "competitor-only" is only claimed when your brand is absent from everything the Actor could read.
Being cited by an AI answer is **never** enough on its own to mark a page as verified.

All verification states:

| State | Meaning | Priority |
|---|---|---|
| `verified_competitor_only` | Fetched; competitor found; brand not found | **high** |
| `competitor_found` | Fetched and competitor found, but the page was too short, truncated or looked paywalled, so your brand's absence can't be trusted | **medium** |
| `brand_found` | Fetched and your brand is on the page | not a gap |
| `no_mentions` | Fetched; neither your brand nor a competitor was found | not a gap |
| `not_verified` | The Actor tried to fetch the page but could not check it. The reason is stored (see below). | **unverified**, if the AI answer that cited it named a competitor and not you |

### What "not verified" means

The Actor could not reliably read the page, so it makes **no claim** about the page's content. Each such page records a reason:

- `http_401` / `http_403`: login required or access denied
- `cloudflare_challenge`: bot protection
- `robots_disallowed`: the site's robots.txt asks crawlers not to fetch that path (the Actor respects this)
- `timeout` / `network_error`
- `not_found`: 404 or 410
- `rate_limited`: 429
- `server_error`: 5xx
- `js_required`: the page is rendered by JavaScript and its HTML contains no readable text
- `paywall`
- `empty_content`
- `unsupported_content_type`: e.g. PDF
- `invalid_url`
- `blocked_host`
- `extraction_failed`
- `verification_cap_reached`: the page was not checked because the "Pages to check" limit was reached

Rows marked "unverified" are leads for a manual check, not findings.

Pages skipped because of the "Pages to check" limit were never fetched, so they are **not** rows in the dataset.
If the AI answer that cited one named a competitor and not you, it is listed once in `OUTPUT.uncheckedSources` (with the competitors that answer named). Raise "Pages to check" to verify them.

### How competitor discovery works

If you list competitors, the Actor uses exactly those. If you don't, it discovers candidates from the AI answers **without any extra AI call**:

1. It takes names that appear in structured positions of the answers: bold text, numbered or bulleted list items, headings and link text.
2. It removes generic headings ("Key Features", "Budget"), cuisine, nationality and language labels ("Japanese", "Bilingual Arabic"), feature and service labels ("UI", "Web Design"), locations, your category words, known source platforms (TripAdvisor, Yelp, Reddit…), publications ("Michelin Guide") and your own brand.
   Names the answers use as places, such as "restaurants in Diriyah" or "the Al Olaya district", are treated as locations and removed as well.
3. It counts how many answers mention each remaining name, and across how many engines. A cited URL that looks like the candidate's own website counts as extra evidence.
4. It returns up to 5 candidates, each with a `confidence` level and a plain-language `reason`:
   - `high`: named in 2 or more answers, and either named by 2 or more engines or its own website was cited;
   - `medium`: named in 2 or more answers;
   - `low`: named in only 1 answer.

These are **businesses the AI named**. They are not guaranteed to be your direct competitors, so review the list.

### How visibility is calculated

There is no secret score. The calculation is:

```
brand visibility      = answers that mention your brand      / successful answers
competitor visibility = answers that mention the competitor / successful answers
```

- Only **discovery questions** are counted: questions that don't name your brand. Two of the standard questions do name it ("alternatives to <brand>", "compare <brand> …"). If they were counted, your brand would appear in nearly every answer and the rate would be inflated. Those answers are still used to find sources.
- Failed answers are excluded from the denominator.
- A mention means the name or domain was matched as whole words. For example, "Nobu" does not match "Nobuko".
- Short or common-word brand names (e.g. "On", "Target") need correct capitalization and a nearby category word. This stops the ordinary word "on" from counting as a mention.
- The results also include a breakdown per engine.
- `topCompetitors` ranks competitors by how often they were observed in AI answers. It is **not** a quality ranking.

### Cost

A run has two separate costs, both billed to your Apify account:

1. **This Actor: $0.20 per completed analysis.** One `analysis-completed` event is charged per run, after the report is saved, and only if at least one AI engine answered. A run where every engine failed is not charged.
2. **The AI searches:** each question on each engine is a run of an official Apify Store Actor, started under your account and billed by that Actor at **your Apify plan's price**. This is on top of the $0.20:

| Engine | Actor | Free plan | Starter | Scale | Business |
|---|---|---|---|---|---|
| Perplexity | [`apify/perplexity-search-scraper`](https://apify.com/apify/perplexity-search-scraper) | $0.20 | $0.013 | $0.010 | $0.009 |
| ChatGPT | [`apify/chatgpt-search-scraper`](https://apify.com/apify/chatgpt-search-scraper) | $0.20 | $0.005 | $0.004 | $0.003 |
| Gemini | [`apify/gemini-search-scraper`](https://apify.com/apify/gemini-search-scraper) | $0.20 | $0.005 | $0.004 | $0.003 |

(Price per question, as listed by those Actors in September 2026. The engine Actors set these prices and can change them.)

Estimated total per run (AI searches + $0.20 analysis):

| Scan | Searches | Free plan | Starter | Scale | Business |
|---|---|---|---|---|---|
| Quick (default: Perplexity, 10 questions) | 10 | $2.00 + $0.20 | $0.13 + $0.20 | $0.10 + $0.20 | $0.09 + $0.20 |
| Full (optional: all three engines, 10 questions) | 30 | **$6.00** + $0.20 | $0.23 + $0.20 | $0.18 + $0.20 | $0.15 + $0.20 |

A small amount of Apify platform usage (Actor start) may also apply.

**On the Free plan, a full analysis costs more than the $5 of monthly free credit.** The quick scan is the default for that reason. In the live test, Perplexity also returned most of the cited sources.
The Actor cannot see your exact plan or remaining credit. Before the first search it shows an estimate as the run's status message: exact on the Free plan, a range on paid plans.
**No OpenAI, Google or Perplexity API key is needed.**

Your **maximum cost per run** covers both: the price of one `analysis-completed` event is reserved first, and the rest is split evenly between the AI engine runs as their own maximum cost. If the limit is too low for even one analysis, the run stops before any AI engine is asked, and nothing is charged. Gemini's Actor requires a maximum cost of at least $0.50 per run: if Gemini's share is lower, Gemini is skipped (the reason is in `OUTPUT.errors`) and the other engines still run.

To prevent runaway costs, a run is capped at 150 AI questions in total (questions × engines × samples). Inputs over any limit fail immediately with a clear message.

### Limitations

- **AI answers vary.** The same question can get a different answer on the next run. Use 2–3 samples for steadier numbers.
- **Answers are scraped from consumer AI products** by the engine Actors listed above. If one of those Actors is down, its engine's answers are marked as failed, and the rest of the run continues.
- **No location parameter.** The engines take no location setting, so the location is written into the question text. Results reflect whatever location context the engines apply on their side.
- **No headless browser.** Pages are fetched with plain HTTP. Pages that need JavaScript, login or bot-protection bypass are reported as `not_verified`; they are never guessed.
- **robots.txt is respected**, and requests to the same site are spaced out. Some sites (for example Reddit and Yelp) often block automated requests, so expect some `not_verified` results.
- **Name matching is deterministic.** Heavily abbreviated names, misspellings or names written in a different script than the one you entered may be missed.
- Source classification uses domain lists and URL/title patterns. Sites it doesn't recognize are classified as `other`.
- The Actor finds opportunities. It does **not** contact anyone.

### Example use case

A restaurant in Riyadh runs the Actor with its name, website and "Restaurant / Saudi Arabia / Riyadh".
With the default quick scan (Perplexity, 10 questions), the results show the restaurant appears in 37.5% of AI answers, while its top competitor appears in all of them.
They also show two pages that were fetched and verified to mention competitors but not the restaurant: a "best restaurants in Riyadh" listicle and a lifestyle news article. Both are cited repeatedly by Perplexity.
Those two pages go to the top of the restaurant's PR outreach list. The Reddit thread the AI cited couldn't be fetched (HTTP 403), so it is listed as "unverified" for a manual check.
To see whether ChatGPT and Gemini cite the same pages, the restaurant can run the optional full scan.

# Actor input Schema

## `brand` (type: `string`):

Your business or brand name, exactly as customers know it. The prefilled value is a fictional demo; replace it with your own.

## `website` (type: `string`):

Your website address, starting with http:// or https:// (e.g. https://yourbrand.com). Used to recognize your own pages and mentions of your domain.

## `category` (type: `string`):

What you are, in a few words, e.g. "restaurant", "CRM software", "dentist".

## `country` (type: `string`):

The country your customers are in. It is written into the questions sent to the AI engines.

## `city` (type: `string`):

Add a city if you serve a local market.

## `competitors` (type: `array`):

Up to 5 competitors. Write a name, a name and website like "Acme | acme.com", or just a website. Leave empty and the Actor will find the businesses AI recommends most.

## `customPrompts` (type: `array`):

Your own questions for the AI engines, up to 25. If empty, 10 standard questions are built from your category and location.

## `engines` (type: `array`):

Which AI engines to ask. Perplexity alone is the quick scan (it cites the most sources). Add ChatGPT and Gemini for the full analysis. Every question on every engine is a separate search billed to your account at your Apify plan's price: $0.20 per search on the Free plan, about $0.002–$0.013 on paid plans.

## `samples` (type: `integer`):

How many times each question is asked per engine. AI answers vary, so more samples give steadier numbers but cost more. A run is capped at 150 AI queries in total (questions × engines × samples).

## `maxPagesToVerify` (type: `integer`):

Maximum number of cited pages to open and check. The most promising pages are checked first.

## Actor input object example

```json
{
  "brand": "Example Restaurant",
  "website": "https://example.com",
  "category": "restaurant",
  "country": "Saudi Arabia",
  "engines": [
    "perplexity"
  ],
  "samples": 1,
  "maxPagesToVerify": 25
}
```

# Actor output Schema

## `sourceGaps` (type: `string`):

No description

## `report` (type: `string`):

No description

## `opportunitiesCsv` (type: `string`):

No description

## `rawResponses` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "brand": "Example Restaurant",
    "website": "https://example.com",
    "category": "restaurant",
    "country": "Saudi Arabia"
};

// Run the Actor and wait for it to finish
const run = await client.actor("nawfaltech/ai-visibility-competitor-source-gaps").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "brand": "Example Restaurant",
    "website": "https://example.com",
    "category": "restaurant",
    "country": "Saudi Arabia",
}

# Run the Actor and wait for it to finish
run = client.actor("nawfaltech/ai-visibility-competitor-source-gaps").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "brand": "Example Restaurant",
  "website": "https://example.com",
  "category": "restaurant",
  "country": "Saudi Arabia"
}' |
apify call nawfaltech/ai-visibility-competitor-source-gaps --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nawfaltech/ai-visibility-competitor-source-gaps"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bavUw7XVhjKiX8bbZ/builds/ZaAIJVdDbyENb7aTn/openapi.json