# SiteVerdict — Keyword Clustering Tool (SERP, AI, GSC) (`siteverdict/siteverdict-keyword-clusterer`) Actor

Turn keyword lists or a Search Console export into topic clusters (one cluster = one page). Free built-in lexical grouping needs no key. Optional SERP-overlap or embedding grouping with your own Serper/SerpApi or OpenAI key. Intent labels, pillar map, cannibalisation flags, CSV export.

- **URL**: https://apify.com/siteverdict/siteverdict-keyword-clusterer.md
- **Developed by:** [Aman Thakran](https://apify.com/siteverdict) (community)
- **Categories:** SEO tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SiteVerdict — Keyword Clustering Tool (SERP, AI, GSC)

**Turn a messy keyword list into a content plan: one cluster = one page.** Paste keywords, a Search Console *Queries* export, or a Semrush/Ahrefs/Keyword Planner CSV. You get topic clusters, a search-intent label per keyword and per cluster, the page type to build, a **pillar → cluster map**, and cannibalisation flags where your site already ranks with several URLs.

Choose the method:

| Method | Needs | Best for |
|---|---|---|
| **SERP overlap** | Your Serper.dev or SerpApi key | The most accurate: keywords that share ≥ 3 of Google's top-10 URLs can rank with one page. SERPs are cached for 30 days, so re-runs don't pay twice. |
| **Embeddings** | Your OpenAI key | Fast semantic grouping of large lists (synonyms, paraphrases). |
| **Lexical** | Nothing (free, offline) | Quick first pass. Groups keywords by shared stemmed topic words. |

### Three ways people use it

1. **Content plan from GSC.** Export *Queries* from Search Console and paste the CSV. Clusters with `cannibalisationRisk` show where 2+ of your pages compete for the same topic.
2. **Agency keyword research.** Cluster 5,000 keywords by SERP overlap. Hand the client `CLUSTERS-CSV`: one row per page to create, with intent, page type and total demand.
3. **AI-labelled briefs.** Turn on AI labels with your own key to get a human label and a one-line brief per cluster.

### Example (sample mode: 17 GSC queries, SERP method, abridged)

```json
{ "type": "cluster", "cluster": 1, "head": "best running shoes", "pillar": "running shoes", "size": 2,
  "keywords": ["best running shoes", "best running shoes for flat feet"], "intent": "commercial",
  "suggestedPageType": "comparison / \"best X\" listicle / review", "totalImpressions": 11700 }
{ "type": "cluster", "head": "marathon training plan", "keywords": ["marathon training plan", "16 week marathon training plan", "beginner marathon training plan"], "intent": "informational" }
```

### Intent labels

Labels are informational, commercial, transactional, navigational (your `brandTerms`) and local ("near me"). They come from modifier rules, and in SERP mode they are refined with SERP features: shopping results → transactional, local pack → local, answer box / People Also Ask → informational. Every keyword row lists the signals used.

### Output

- Dataset: **Clusters** (one page each), **Keywords**, **Summary** (pillars, intent mix, cannibalisation risks, SERP calls vs cache hits).
- Key-value store: `CLUSTER-MAP` (HTML pillar/cluster map), `KEYWORDS-CSV`, `CLUSTERS-CSV`, `SUMMARY`.

### Pricing (pay per event)

| Event | Price | When |
|---|---|---|
| `keyword-clustered` | $0.001 | Each unique keyword clustered (any method) |
| `ai-cluster-label` | $0.002 | Each cluster labelled + briefed by your own AI key |

Examples: 1,000 keywords ≈ **$1.00**. 10,000 keywords ≈ **$10**. With SERP mode, your SERP provider bills you separately (e.g. Serper.dev ≈ $0.30–1 per 1,000 searches at the time of writing). Embeddings cost cents per 10,000 keywords on your OpenAI account. Sample mode is free.

#### What happens when your max total charge is too small

- **Keywords come first.** Before clustering, the run works out how many keywords your max total charge covers. Clustering looks at the whole set, so with a short budget the run clusters only the **top N keywords by demand** (volume, or impressions for GSC) that fit, and charges exactly those N. Clusters are therefore built from those N keywords only.
- Every keyword that did not fit gets its own dataset row (`type: "keyword-skipped"`, `status: "not_processed_max_charge"`). These rows are **free**. The run status ends with "N not processed (max total charge reached)" and `SUMMARY.notProcessed` = N.
- **AI labels use only what is left after the keywords.** Example: max $1.00 with 1,000 keywords and AI labels on → 1,000 keywords clustered ($1.00), no AI labels; the status says how many clusters were not labelled. Max $1.10 → 1,000 keywords + 50 AI labels. The AI provider is not called for labels that can't be paid.
- If your budget is used exactly (e.g. 50 keywords with a max of $0.05), the run is complete and is not reported as stopped.
- `SUMMARY.chargedEvents` lists exactly what was charged.

### FAQ

**Do you scrape Google?** No. SERP data comes only from the official SERP API you choose, on your own account.

**Which threshold should I use?** SERP overlap 3/10 is a common industry default; use 4–5 for tighter clusters. Embeddings default to 0.82 cosine similarity.

**Can I use a Search Console export?** Yes. Columns (Top queries, Clicks, Impressions, Position, and Page if present) are detected automatically. Impressions are used as demand when there is no volume.

### Limits

- 20,000 keywords per run (5,000 in SERP mode).
- AI labels: at most 500 clusters per run (the 500 with the highest demand).
- CSV files up to 20 MB.
- Lexical clustering is English-oriented (light stemming). For other languages, use SERP or embeddings.

### Sources & compliance

- Inputs are your own keyword lists and exports. SERPs come from Serper.dev (serper.dev/terms) or SerpApi (serpapi.com/legal) on your API key. Embeddings and labels come from OpenAI / Anthropic on your API key. Terms were checked on 11 Oct 2026; you are responsible for your provider account's terms.
- All keys are Apify secret inputs. They are redacted from logs and output, and sent only to the provider they belong to. Cached SERPs never contain keys.
- No personal data is processed. Search queries from Search Console are aggregated by Google. Under UK GDPR/PECR, avoid uploading lists that contain personal data.
- A CSV URL is fetched once, with an SSRF guard (public https/http only; private, loopback, link-local and cloud-metadata addresses are refused; redirects are not followed, so give the final URL).
- **Lexical mode needs no key and makes no network requests** (except the optional CSV URL). SERP mode and embeddings **need your own API key**; nothing is clustered by SERP or embeddings without one (the run falls back to lexical and says so in the log).
- Hosts called, and only when you turn the matching option on: `google.serper.dev` (Serper.dev) or `serpapi.com` (SerpApi) for SERPs; `api.openai.com` for embeddings or OpenAI labels; `api.anthropic.com` for Anthropic labels; your CSV URL. Those providers bill you directly under their own terms.
- Support: hello@webrevenuescan.com.

# Changelog

This Actor's version history is a separate document: https://apify.com/siteverdict/siteverdict-keyword-clusterer/changelog.md

# Actor input Schema

## `keywords` (type: `array`):

One per line. Optional volume after a comma or tab: 'best running shoes, 4400'. Max 20,000.

## `csvText` (type: `string`):

Search Console 'Queries' export, Semrush/Ahrefs export or any CSV with a keyword column. Columns are detected automatically.

## `csvUrl` (type: `string`):

Public https:// link to a CSV file (e.g. a published Google Sheet as CSV).

## `method` (type: `string`):

auto = SERP if a SERP key is given, else embeddings if an OpenAI key is given, else lexical.

## `serpProvider` (type: `string`):

Your SERP API account.

## `serpApiKey` (type: `string`):

Your own Serper.dev or SerpApi key (one search per keyword; results cached 30 days). Stored encrypted as an Apify secret, never logged or written to the output.

## `country` (type: `string`):

2-letter country code for SERPs, e.g. gb, us, de.

## `language` (type: `string`):

Language code, e.g. en, de.

## `serpOverlap` (type: `integer`):

How many of the top-10 URLs two keywords must share to be one page. 3 is a common default; 4-5 = tighter clusters.

## `useSerpCache` (type: `boolean`):

Cache SERPs in the named store 'siteverdict-keyword-serp-cache' so re-runs do not pay your SERP API twice.

## `openaiApiKey` (type: `string`):

Your own OpenAI key for embedding clustering. Stored encrypted as an Apify secret, never logged or written to the output.

## `embeddingModel` (type: `string`):

OpenAI embedding model.

## `similarityThreshold` (type: `number`):

Cosine similarity to the cluster head (0.5-0.99). Higher = tighter clusters.

## `lexicalThreshold` (type: `number`):

Weighted word overlap (0.1-1). Higher = tighter clusters.

## `brandTerms` (type: `array`):

Your brand names (and competitors') so branded keywords get navigational intent.

## `minVolume` (type: `integer`):

Drop keywords below this demand.

## `aiLabels` (type: `boolean`):

Name each cluster and write a one-line page brief with your own Anthropic/OpenAI key.

## `aiProvider` (type: `string`):

Which API your key belongs to.

## `aiApiKey` (type: `string`):

Your own API key (we never use ours). Stored encrypted as an Apify secret, never logged or written to the output.

## `aiModel` (type: `string`):

Leave empty for a fast, low-cost default (claude-haiku-4-5 / gpt-4.1-mini).

## `sampleMode` (type: `boolean`):

Run on a built-in demo with planted problems. No network requests and nothing charged. Use it to see the output format.

## Actor input object example

```json
{
  "keywords": [
    "best running shoes",
    "running shoes for women",
    "trail running shoes",
    "how to choose running shoes",
    "marathon training plan",
    "beginner marathon training plan"
  ],
  "method": "auto",
  "serpProvider": "serper",
  "country": "gb",
  "language": "en",
  "serpOverlap": 3,
  "useSerpCache": true,
  "embeddingModel": "text-embedding-3-small",
  "similarityThreshold": 0.82,
  "lexicalThreshold": 0.5,
  "minVolume": 0,
  "aiLabels": false,
  "aiProvider": "anthropic",
  "sampleMode": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `files` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "best running shoes",
        "running shoes for women",
        "trail running shoes",
        "how to choose running shoes",
        "marathon training plan",
        "beginner marathon training plan"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("siteverdict/siteverdict-keyword-clusterer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "keywords": [
        "best running shoes",
        "running shoes for women",
        "trail running shoes",
        "how to choose running shoes",
        "marathon training plan",
        "beginner marathon training plan",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("siteverdict/siteverdict-keyword-clusterer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "best running shoes",
    "running shoes for women",
    "trail running shoes",
    "how to choose running shoes",
    "marathon training plan",
    "beginner marathon training plan"
  ]
}' |
apify call siteverdict/siteverdict-keyword-clusterer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,siteverdict/siteverdict-keyword-clusterer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zLkq7RiMKCr5Awt03/builds/gdUKncanB0m9x7g9n/openapi.json