Keyword Gap (Semrush)

Compare your domain against competitors and get every keyword they rank for that you don't — plus keywords where they outrank you. Sorted by search volume, exported to Excel.

How it works

your domain ─▶ domain_organic ─▶ your keyword map
competitors ─▶ domain_organic ─▶ compare each keyword
                                   ├─ not in your map        → gap_type "missing"
                                   ├─ their position better  → gap_type "weak"
                                   └─ you rank as well/better → skipped
                                 dedup across rivals (best position wins)
                                 sort by volume ─▶ dataset + Excel

⚠️ Billing model (important)

Semrush bills ~10 units for every row fetched — yours and each competitor's — not just the gaps we surface. So this actor charges per keyword analysed (keyword_analyzed), which keeps revenue aligned with cost regardless of how large the gap turns out to be.

Example: 1 domain + 2 competitors × 1,000 keywords = 3,000 analysed = 3,000 billable events, which might produce anywhere from 200 to 2,800 gap rows.

Charging per gap row instead would be dangerous: comparing two near-identical domains returns almost no gaps while still costing full price in units.

Safety rails

• Charge-limit guard — respects the user's max cost per run; stops before spending units it cannot bill.
• maxKeywordsPerDomain — bounds cost per domain.
• Server-side filters (minVolume / maxPosition) applied by Semrush before billing, so narrowing genuinely costs less.
• Units pre-flight — logs the estimate and your balance.
• Fatal-error stop — bad key/quota halts the run; delivered rows are kept.

Setup

Set SEMRUSH_API_KEY as a secret environment variable on the actor.

Run locally

cd semrush-keyword-gap
npm install
$env:SEMRUSH_API_KEY = "your-semrush-api-key"   # PowerShell
node src/main.js

Example input:

{
  "yourDomain": "moz.com",
  "competitorDomains": ["ahrefs.com", "semrush.com"],
  "database": "us",
  "maxKeywordsPerDomain": 1000,
  "minVolume": 1000,
  "gapTypes": ["missing", "weak"]
}

Input

Field	Meaning
yourDomain	the domain you want gaps for (required)
competitorDomains / competitorsText	who to compare against
database	country market (us, uk, de…)
maxKeywordsPerDomain	rows pulled per domain — drives depth and cost
minVolume / maxPosition	server-side filters (also reduce cost)
gapTypes	missing, weak, or both
excludeKeywords	drop keywords containing these terms (applied server-side → also cheaper)
xlsxOutput	save formatted Excel (default on)
semrushApiKey	optional key override

Output

{
  "keyword": "keyword research",
  "gap_type": "weak",
  "volume": 1830000,
  "cpc": 4.86,
  "your_position": 31,
  "best_competitor": "semrush.com",
  "their_position": 1,
  "their_url": "https://www.semrush.com/...",
  "competitors_ranking": ["semrush.com", "ahrefs.com"],
  "database": "us"
}

gap_type: "missing" rows have your_position: null.

Publish + pay-per-event

apify login
apify push

Publication ▸ Monetization ▸ Pay per event — delete the auto-added apify-default-dataset-item "Result" event (the code charges explicitly), then add:

Event name	Title	Suggested price
keyword_analyzed	Keyword analysed	$0.002 ($2 / 1,000)

Filtering out noise

Large domains rank for plenty of unrelated terms (Semrush ranks for random brand names because of their traffic-checker pages). Use excludeKeywords to drop them:

"excludeKeywords": ["xnxx", "simpcity", "casino", "porn"]

Up to 10 terms are pushed into Semrush's display_filter as negative -|Ph|Co|<term> clauses, so excluded rows are never fetched and never billed; anything beyond that is filtered client-side as a safety net.

Roadmap

1. Intent tagging — classify each gap keyword with a cheap LLM pass.
2. Cluster gaps by ranking URL — see which of their pages to compete with.
3. Scheduled runs — track gap closure over time.