# Content Gap By SERP Pages Auditor (`junipr/content-gap-by-serp-pages-auditor`) Actor

Compare owned pages against top-ranking SERP pages to identify missing subtopics, headings, FAQs, entities, media signals, and content-format gaps

- **URL**: https://apify.com/junipr/content-gap-by-serp-pages-auditor.md
- **Developed by:** [junipr](https://apify.com/junipr) (community)
- **Categories:** SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $11.50 / 1,000 owned page analyzeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Content Gap By SERP Pages Auditor

Compare an owned page with supplied competitor-page evidence for the same search query. The Actor extracts visible topics, question headings, content formats, and structured-data entity types, then emits only the signals present on competitors and absent from the owned page.

This is useful for SEO content refreshes, editorial briefs, landing-page research, and repeatable competitor inventories. It does not claim that a missing signal is automatically worth adding. Each row includes the source URLs, the compared evidence, a transparent frequency, and a bounded priority score so an editor can make the final decision.

### What the Actor checks

- meaningful words from titles, meta descriptions, and visible headings;
- visible questions in headings, summaries, definition terms, and buttons;
- observable formats such as lists, tables, images, video or embeds, and FAQs;
- JSON-LD `@type` values that can be parsed from supplied pages;
- the number of accepted competitor pages that contain each missing signal;
- whether every query has at least one owned page and one competitor page with usable content.

No gap is invented when a source is empty, blocked, or missing. Access-denied and human-verification pages produce a diagnostic row instead of a content recommendation.

### Input

Use one of two evidence paths:

1. `ownedUrls` and `competitorUrls` fetch public pages. Add `targetQueries` in corresponding order when needed.
2. `serpResults` accepts structured query records with `ownedUrl`, `ownedHtml`, and a `competitors` array. This path is ideal when another search workflow already captured page HTML or snippets.

`htmlInputs` can also carry records with `sourceUrl`, `html`, `role` (`owned` or `competitor`), and `query`.

| Field | Purpose | Default / cap |
| --- | --- | --- |
| `targetQueries` | Query labels used to group comparisons | one bounded default query; `maxQueries` 10 |
| `ownedUrls` | Public owned-page URLs | empty; `maxOwnedPages` 5 per query |
| `competitorUrls` | Public competitor-page URLs | empty; `maxCompetitorPages` 10 per query |
| `serpResults` | Query-linked owned and competitor evidence | empty array |
| `gapDimensions` | `topics`, `questions`, `formats`, and/or `entities` | all four |
| `minCompetitorFrequency` | Minimum accepted competitors showing a signal | 1, maximum 20 |
| `maxSectionsPerPage` | Maximum gap rows emitted per query | 25, maximum 100 |
| `allowedDomains` | Optional hostname allowlist for URL fetches | empty |
| `maxPages` | Maximum public pages fetched in one run | 1 by default, maximum 25 |
| `includeEvidence` | Keep evidence excerpts in dataset rows | `true` |
| `includeRawSnapshots` | Store accepted raw HTML snapshots | `false` |
| `maxChargeUsd` | Stop before the next event would exceed this amount | 35 USD maximum |
| `includeReport` | Write report artifacts | `true` |

Example using captured SERP evidence:

```json
{
  "serpResults": [
    {
      "query": "project management software",
      "ownedUrl": "https://owned.example/project-management",
      "ownedHtml": "<html><title>Project management</title><h1>Task planning</h1></html>",
      "competitors": [
        {
          "url": "https://competitor.example/workload",
          "html": "<html><title>Workload management</title><h1>Capacity planning</h1><h2>How do teams allocate resources?</h2></html>"
        }
      ]
    }
  ],
  "targetQueries": ["project management software"],
  "gapDimensions": ["topics", "questions"],
  "maxSectionsPerPage": 20,
  "maxChargeUsd": 5
}
```

The Store default intentionally contains no page evidence. It finishes quickly with a nonempty `sources_required` diagnostic that explains what to provide.

### Dataset output

Every dataset item has the same schema. Important fields include:

- `targetQuery`, `ownedUrl`, and `competitorUrl` identify the comparison;
- `gapType` is `topic`, `question`, `format`, `entity`, or `none`;
- `missingTopic` is the exact normalized signal;
- `competitorEvidence` and `ownedEvidence` preserve bounded source excerpts;
- `priorityScore` combines competitor frequency with the signal class;
- `issueCode`, `severity`, `evidenceSnippet`, and `recommendation` explain the decision;
- `sourceUrl`, `canonicalUrl`, `httpStatus`, and `scannedAt` preserve provenance.

When no configured gap meets the threshold, the Actor can emit a `none` baseline. Missing or blocked evidence produces a diagnostic with null gap fields.

### Reports

When `includeReport` is enabled, key-value storage receives:

- a Markdown summary;
- a CSV containing every dataset row;
- a JSON list of issue rows;
- bounded evidence excerpts;
- run metadata with counts, consumed inputs, strategy identifier, and pricing mode.

Raw page snapshots are separate and opt-in. This keeps ordinary runs compact and avoids retaining more source content than necessary.

### Pay-per-event pricing

| Event | Price (USD) | When charged |
| --- | ---: | --- |
| `actor-start` | 0.035 | after the run starts successfully |
| `owned-page-analyzed` | 0.0115 | once for each source/query unit processed |
| `competitor-source-processed` | 0.0115 | before emitting a row backed by accepted competitor evidence |
| `content-gap-emitted` | 0.0115 | before each paid gap dataset item |
| `executive-report-generated` | 0.4000 | before report artifacts are written |

Apify platform usage pass-through is off. The Actor checks `maxChargeUsd` before every event and stops cleanly before writing an output whose charge cannot be accepted.

### Responsible use and limits

Only public HTTP and HTTPS URLs are accepted. Loopback and private IPv4 targets are rejected, an optional domain allowlist can narrow fetches, and general discovered-link crawling is disabled. Sitemap URLs remain subject to the same page cap. The Actor does not bypass authentication, paywalls, robots controls, or human-verification challenges.

The result is an evidence inventory, not a promise of search ranking or editorial performance. Review recommendations for relevance, accuracy, copyright, and brand requirements before publishing content changes.

# Actor input Schema

## `ownedUrls` (type: `array`):

Public owned-page URLs to compare.

## `competitorUrls` (type: `array`):

Public competitor-page URLs for the same target queries.

## `serpResults` (type: `array`):

Query-linked owned and competitor page snapshots. Each record may include query, ownedUrl, ownedHtml, and a competitors array with url, title, snippet, html, or text.

## `targetQueries` (type: `array`):

Queries used to group owned and competitor evidence.

## `gapDimensions` (type: `array`):

Observable page-signal groups to compare.

## `maxOwnedPages` (type: `integer`):

Maximum owned pages to compare for each query.

## `maxCompetitorPages` (type: `integer`):

Maximum competitor pages to compare for each query.

## `maxQueries` (type: `integer`):

Maximum search queries or topics to evaluate in one run.

## `maxSectionsPerPage` (type: `integer`):

Maximum page sections to inspect per source page.

## `minCompetitorFrequency` (type: `integer`):

Minimum number of accepted competitor pages that must show a signal before it can be emitted as a gap.

## `startUrls` (type: `array`):

Optional public URLs. Use ownedUrls and competitorUrls when roles matter.

## `sitemapUrls` (type: `array`):

Optional public XML sitemap URLs. Accepted page URLs remain bounded by maxPages.

## `htmlInputs` (type: `array`):

Optional captured HTML records with sourceUrl, html or currentHtml, role, and query.

## `allowedDomains` (type: `array`):

Optional hostname allowlist for fetched pages.

## `maxPages` (type: `integer`):

Maximum pages to fetch in one run.

## `maxDepth` (type: `integer`):

General link discovery is disabled.

## `includeEvidence` (type: `boolean`):

Include short source evidence snippets in output rows.

## `includeRawSnapshots` (type: `boolean`):

Opt in to raw HTML artifacts in key-value storage.

## `requestDelayMs` (type: `integer`):

Delay in milliseconds between outbound page requests.

## `timeoutMs` (type: `integer`):

Maximum time in milliseconds to wait for a page request.

## `userAgentMode` (type: `string`):

User agent profile to use for public page requests.

## `maxChargeUsd` (type: `number`):

Maximum estimated PPE spend before the actor exits gracefully.

## `includeReport` (type: `boolean`):

Write the markdown summary report to the run key-value store.

## Actor input object example

```json
{
  "ownedUrls": [],
  "competitorUrls": [],
  "serpResults": [],
  "targetQueries": [
    "project management software"
  ],
  "gapDimensions": [
    "topics",
    "questions",
    "formats",
    "entities"
  ],
  "maxOwnedPages": 5,
  "maxCompetitorPages": 10,
  "maxQueries": 10,
  "maxSectionsPerPage": 25,
  "minCompetitorFrequency": 1,
  "startUrls": [],
  "sitemapUrls": [],
  "htmlInputs": [],
  "allowedDomains": [],
  "maxPages": 1,
  "maxDepth": 0,
  "includeEvidence": true,
  "includeRawSnapshots": false,
  "requestDelayMs": 250,
  "timeoutMs": 15000,
  "userAgentMode": "standard",
  "maxChargeUsd": 35,
  "includeReport": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `report` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("junipr/content-gap-by-serp-pages-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("junipr/content-gap-by-serp-pages-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call junipr/content-gap-by-serp-pages-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,junipr/content-gap-by-serp-pages-auditor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/AE6h6YuGaDmUmCtCs/builds/fz7uKv5ecfoSUfxEp/openapi.json