# SERP Topic Gap Monitor (`joeslade/serp-topic-gap-monitor`) Actor Find the topics your competitors rank for that your site is missing. This SEO content gap analyzer accepts pre-fetched SERP data, extracts topics from competitor pages, and returns a scored gap report for content planning and competitive analysis. - **URL**: https://apify.com/joeslade/serp-topic-gap-monitor.md - **Developed by:** [Joe Slade](https://apify.com/joeslade) (community) - **Categories:** SEO tools - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## SERP Topic Gap Monitor Find the topics your competitors rank for that your site is missing. This SEO content gap analyzer takes pre-fetched SERP data, extracts topics from competitor pages, and returns a scored gap report you can act on immediately. ### Who is this for? - **SEO strategists** building data-driven content calendars - **Content teams** deciding what to write next based on competitive gaps - **Agencies** running repeatable competitor content analysis across client portfolios - **Growth marketers** prioritizing keyword clusters by competitive opportunity ### What it does Give it your domain and a set of search results. The Actor compares what your competitors rank for against what your site covers, then returns a ranked list of topic gaps -- the content your competitors publish that you have not. You control the SERP data source. This Actor processes structured search results that you provide, rather than fetching them itself. That means predictable costs (no surprise API calls), full control over which keywords and markets you analyze, and compatibility with any SERP data source -- Google SERP API actors, your own scraping pipeline, or exported spreadsheets. ### Use cases - **SEO content planning** -- Find topics your competitors rank for that you have not covered yet - **Content gap analysis** -- Identify which keyword clusters have the weakest coverage on your domain - **Competitor content analysis** -- See which competitor pages cover topics you are missing - **Editorial prioritization** -- Rank content gaps by severity to focus on the highest-impact topics first ### How it works The Actor runs a four-step deterministic pipeline. Identical input always produces identical output. 1. **Tokenize** -- Page content (or title + snippet when no content is available) is lowercased, stripped of punctuation, and filtered through a standard English stopword list. Tokens under 3 characters are removed. 2. **Extract topics** -- For each keyword's SERP results, tokens that appear on 2 or more competitor pages are identified as topics. Your target domain's pages are checked for coverage but do not count toward the competitor threshold. 3. **Detect gaps** -- Topics are merged across keywords. Competitor URLs are deduplicated. If your domain covers a topic in any keyword's results, it is marked as covered. 4. **Score and rank** -- Each gap receives a score: `gapScore = uniqueCompetitorPages / totalUniqueCompetitorPages`, clamped to [0, 1] and rounded to 2 decimal places. Gaps are sorted by score (highest first), then alphabetically by topic. ### Input | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `targetDomain` | string | Yes | -- | Domain to evaluate (e.g., `myblog.com`). Matches the hostname and subdomains (e.g., `www.myblog.com`, `blog.myblog.com`). | | `serpResults` | array | Yes | -- | Pre-fetched SERP data. Each entry represents one keyword's search results. See [Input format](#input-format) below. | | `locale` | string | No | `en` | Locale of the SERP data (e.g., `en`, `de`, `ja`). Stored in output metadata for downstream tracking. Set this when analyzing non-English markets so your reports stay labeled correctly. | | `country` | string | No | `us` | Country code for the SERP data (e.g., `us`, `gb`, `de`). Stored in output metadata for downstream tracking. Useful when you run gap analyses across multiple markets. | | `minGapScore` | number | No | `0` | Minimum gap score threshold (0--1). Gaps scoring below this are excluded from output. Raise this (e.g., `0.3`) to filter out low-signal gaps and focus on topics where many competitors outrank you. | #### Input format Each entry in `serpResults` represents one keyword's organic search results: ```json { "keyword": "content marketing strategy", "organicResults": [ { "url": "https://competitor.com/guide", "title": "Ultimate Content Marketing Guide", "snippet": "Learn how to build a content marketing strategy...", "pageContent": "Full extracted text of the page..." } ] } ```` - `url` and `title` are required for each organic result - `snippet` is optional -- used as fallback text when `pageContent` is missing - `pageContent` is optional but strongly recommended -- without it, topic extraction relies only on titles and snippets, which limits accuracy #### Example input ```json { "targetDomain": "myblog.com", "serpResults": [ { "keyword": "content marketing strategy", "organicResults": [ { "url": "https://myblog.com/strategy", "title": "Content Strategy Guide", "pageContent": "content marketing strategy planning editorial calendar audience research" }, { "url": "https://competitor-a.com/guide", "title": "Ultimate Content Marketing Guide", "pageContent": "content marketing strategy seo optimization keyword research analytics tracking performance metrics" }, { "url": "https://competitor-b.com/tips", "title": "Marketing Tips", "pageContent": "content marketing strategy distribution channels social media analytics tracking engagement" } ] } ], "minGapScore": 0.3 } ``` ### Output The Actor stores its output in the default key-value store under the `OUTPUT` key as a single JSON object. Use this data to decide what content to create, which topics to expand, and where your competitors have the strongest coverage advantage. #### Output fields | Field | Type | Description | |-------|------|-------------| | `targetDomain` | string | The domain that was analyzed | | `keywordsAnalyzed` | number | Number of keywords processed | | `totalGaps` | number | Number of gaps in the output (after `minGapScore` filtering) | | `gaps` | array | Ranked list of topic gaps (see below) | | `serpSummary` | array | Per-keyword summary of SERP data processed | | `meta` | object | Processing metadata (`locale`, `country`, `minGapScore`) | | `notes` | array | Informational notes (e.g., filter warnings) | #### Gap object Each gap in the `gaps` array contains: | Field | Type | Description | |-------|------|-------------| | `topic` | string | The topic token (normalized, lowercase) | | `gapScore` | number | Coverage ratio (0--1). Higher means more competitors cover this topic. | | `competitorCoverage` | number | Number of unique competitor pages covering this topic | | `competitorUrls` | string\[] | URLs of competitor pages covering this topic (sorted) | | `targetCoverage` | boolean | Whether your domain covers this topic in any keyword's results | | `relatedKeywords` | string\[] | Keywords where this topic appeared (sorted) | #### SERP summary object Each entry in `serpSummary` contains: | Field | Type | Description | |-------|------|-------------| | `keyword` | string | The keyword analyzed | | `resultsFetched` | number | Total organic results provided for this keyword | | `pagesFetched` | number | Results that included `pageContent` | | `targetDomainRank` | number | null | Position of target domain in results (1-indexed), or `null` if not found | #### Example output ```json { "targetDomain": "myblog.com", "keywordsAnalyzed": 2, "totalGaps": 3, "gaps": [ { "topic": "analytics", "gapScore": 0.8, "competitorCoverage": 4, "competitorUrls": [ "https://competitor-a.com/guide", "https://competitor-a.com/seo", "https://competitor-b.com/tips", "https://competitor-c.com/blog" ], "targetCoverage": false, "relatedKeywords": ["content marketing strategy", "seo keyword research"] }, { "topic": "tracking", "gapScore": 0.8, "competitorCoverage": 4, "competitorUrls": [ "https://competitor-a.com/guide", "https://competitor-a.com/seo", "https://competitor-b.com/tips", "https://competitor-c.com/blog" ], "targetCoverage": false, "relatedKeywords": ["content marketing strategy", "seo keyword research"] }, { "topic": "strategy", "gapScore": 0.6, "competitorCoverage": 3, "competitorUrls": [ "https://competitor-a.com/guide", "https://competitor-b.com/tips", "https://competitor-c.com/blog" ], "targetCoverage": true, "relatedKeywords": ["content marketing strategy"] } ], "serpSummary": [ { "keyword": "content marketing strategy", "resultsFetched": 4, "pagesFetched": 4, "targetDomainRank": 1 }, { "keyword": "seo keyword research", "resultsFetched": 3, "pagesFetched": 3, "targetDomainRank": 3 } ], "meta": { "locale": "en", "country": "us", "minGapScore": 0 }, "notes": [] } ``` ### How to read the results - **High `gapScore` + `targetCoverage: false`** -- Strong signal to create content. Many competitors cover this topic and you do not. - **High `gapScore` + `targetCoverage: true`** -- You cover this topic, but competitors cover it more broadly. Consider expanding your content. - **Low `gapScore`** -- Only a few competitors mention this topic. Lower priority unless strategically important to your roadmap. - **`relatedKeywords`** -- Shows which search queries surface this gap. More keywords means more search visibility at stake. ### Error handling | Error | Cause | Fix | |-------|-------|-----| | `No input provided.` | Actor received no input | Provide input via the Apify console or API | | `Input must include "targetDomain" and a non-empty "serpResults" array.` | Missing or empty required fields | Ensure `targetDomain` is a non-empty string and `serpResults` has at least one entry | Malformed URLs in `organicResults` are handled gracefully -- they are treated as competitor pages (not matched as target domain) and do not crash the Actor. ### Running locally ```bash ## Install dependencies npm install ## Set up input -- edit storage/key_value_stores/default/INPUT.json ## Then run: npm run start:dev ## Run tests npm test ## Build for production npm run build ``` ### Architecture This Actor is designed as a composable building block in a larger SEO pipeline. Each processing step is a pure function with no side effects. ``` serpResults --> tokenize --> extractTopics --> detectGaps --> scoreAndRank --> output ``` **Why this matters for your workflow:** - **Composable** -- Pair it with any SERP-fetching Actor upstream and any reporting tool downstream. No vendor lock-in on data sources. - **Reusable data** -- Fetch SERP data once, then run multiple gap analyses against different target domains without additional API costs. - **Deterministic** -- Same input always produces the same output. Safe to use in automated pipelines, scheduled runs, and CI/CD workflows. - **Testable** -- Pure-function internals mean you can validate behavior without network access. # Actor input Schema ## `targetDomain` (type: `string`): Domain to evaluate gaps for (e.g. "example.com"). The actor checks which topics this domain is missing relative to competitors. ## `serpResults` (type: `array`): Pre-fetched SERP data. Each entry represents one keyword's search results with page content already extracted. ## `locale` (type: `string`): Locale of the SERP data, e.g. "en", "de", "ja". Stored in output metadata for downstream tracking. Set this when analyzing non-English markets. ## `country` (type: `string`): Country code for the SERP data, e.g. "us", "gb", "de". Stored in output metadata for downstream tracking. Useful when running gap analyses across multiple markets. ## `minGapScore` (type: `number`): Minimum gap score (0-1) to include in output. Raise this (e.g. 0.3) to filter out low-signal gaps and focus on topics where many competitors outrank you. ## Actor input object example ```json { "locale": "en", "country": "us", "minGapScore": 0 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("joeslade/serp-topic-gap-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("joeslade/serp-topic-gap-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call joeslade/serp-topic-gap-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=joeslade/serp-topic-gap-monitor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "SERP Topic Gap Monitor", "description": "Find the topics your competitors rank for that your site is missing. This SEO content gap analyzer accepts pre-fetched SERP data, extracts topics from competitor pages, and returns a scored gap report for content planning and competitive analysis.", "version": "0.1", "x-build-id": "mechiAjKKM4jVUaN4" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/joeslade~serp-topic-gap-monitor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-joeslade-serp-topic-gap-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/joeslade~serp-topic-gap-monitor/runs": { "post": { "operationId": "runs-sync-joeslade-serp-topic-gap-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/joeslade~serp-topic-gap-monitor/run-sync": { "post": { "operationId": "run-sync-joeslade-serp-topic-gap-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "targetDomain", "serpResults" ], "properties": { "targetDomain": { "title": "Target domain", "minLength": 1, "maxLength": 200, "type": "string", "description": "Domain to evaluate gaps for (e.g. \"example.com\"). The actor checks which topics this domain is missing relative to competitors." }, "serpResults": { "title": "SERP results", "minItems": 1, "maxItems": 20, "type": "array", "description": "Pre-fetched SERP data. Each entry represents one keyword's search results with page content already extracted.", "items": { "type": "object", "required": [ "keyword", "organicResults" ], "properties": { "keyword": { "title": "Keyword", "type": "string", "description": "The search keyword this result set corresponds to." }, "organicResults": { "title": "Organic results", "type": "array", "description": "Top organic search results for this keyword.", "maxItems": 20, "items": { "type": "object", "required": [ "url", "title" ], "properties": { "url": { "title": "URL", "type": "string", "description": "URL of the search result." }, "title": { "title": "Title", "type": "string", "description": "Title of the search result." }, "snippet": { "title": "Snippet", "type": "string", "description": "Search result snippet text." }, "pageContent": { "title": "Page content", "type": "string", "description": "Extracted text content of the page (plain text, no HTML). If omitted, only title and snippet are used for topic extraction." } } } } } } }, "locale": { "title": "Locale", "type": "string", "description": "Locale of the SERP data, e.g. \"en\", \"de\", \"ja\". Stored in output metadata for downstream tracking. Set this when analyzing non-English markets.", "default": "en" }, "country": { "title": "Country", "type": "string", "description": "Country code for the SERP data, e.g. \"us\", \"gb\", \"de\". Stored in output metadata for downstream tracking. Useful when running gap analyses across multiple markets.", "default": "us" }, "minGapScore": { "title": "Minimum gap score", "minimum": 0, "maximum": 1, "type": "number", "description": "Minimum gap score (0-1) to include in output. Raise this (e.g. 0.3) to filter out low-signal gaps and focus on topics where many competitors outrank you.", "default": 0 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```