Competitor Keyword Spy (Semrush)

Enter a domain → get every keyword it ranks for, with position, search volume, CPC, traffic share and the exact ranking URL. Optionally discover its organic competitors and analyse them too. Excel export included.

Why this actor

The single most-requested SEO job is "what does my competitor rank for?". This turns one domain into thousands of billable rows — a much better revenue shape than tools where the user supplies the keywords.

How it works

domain(s) ─▶ (optional) domain_organic_organic → competitors
          └▶ domain_organic (sorted by traffic, server-side filtered)
                 └▶ dataset + Excel  ·  charge per delivered row

• domain_organic — ranking keywords. ~10 units/row.
• domain_organic_organic — organic competitors.
• Server-side filters (minVolume / maxPosition) are applied by Semrush before billing, so you don't pay units for rows the user would discard.

API facts verified against a live key

Fact	Result
display_limit	works up to at least 2000 rows in one call
display_offset	not supported on domain_organic (returns HTTP 400) — use a bigger display_limit, not paging
display_filter	works — +|Nq|Gt|999, +|Po|Lt|11 verified to actually filter
display_sort=tr_desc	returns highest-traffic keywords first
Cost	10 units per returned row

Safety rails

• Domain clean + dedup — https://www.ahrefs.com/, ahrefs.com → one target.
• maxKeywordsPerDomain — hard per-domain cap.
• Units pre-flight — logs the estimate and your balance before spending.
• Charge-limit guard — respects the user's max-cost-per-run so you never spend Semrush units you can't bill.
• Fatal-error stop — bad key/quota halts the run; rows already delivered are kept.

Setup

Set SEMRUSH_API_KEY as a secret environment variable on the actor (Apify Console → Settings → Environment variables). Users can override with the semrushApiKey input if you prefer bring-your-own-key.

Run locally

cd semrush-competitor-keywords
npm install
# PowerShell:
$env:SEMRUSH_API_KEY = "your-semrush-api-key"
node src/main.js     # reads storage/key_value_stores/default/INPUT.json

Example input:

{
  "domains": ["ahrefs.com", "moz.com"],
  "database": "us",
  "maxKeywordsPerDomain": 1000,
  "minVolume": 1000,
  "maxPosition": 10,
  "discoverCompetitors": true,
  "competitorsPerDomain": 5
}

Input

Field	Meaning
domains / domainsText	domains to analyse
database	country market (us, uk, de…)
maxKeywordsPerDomain	per-domain row cap
minVolume / maxPosition	server-side filters (also reduce cost)
excludeKeywords	drop keywords containing these terms (server-side → also cheaper)
discoverCompetitors	also return each domain's organic competitors
competitorsPerDomain	how many competitors per seed
includeCompetitorKeywords	also pull keywords for discovered competitors (multiplies cost)
xlsxOutput	save formatted Excel (default on)
semrushApiKey	optional key override

Output

Keyword row:

{
  "domain": "ahrefs.com",
  "keyword": "search operators",
  "position": 1,
  "volume": 1220000,
  "cpc": 2.02,
  "url": "https://ahrefs.com/blog/google-advanced-search-operators/",
  "traffic_pct": 11.45,
  "database": "us"
}

Competitor row (when enabled) carries type: "competitor", seed_domain, competitor_domain, relevance, organic_keywords, organic_traffic.

The Excel file has two sheets: Keywords and Competitors.

Publish + pay-per-event

apify login
apify push

Then Publication ▸ Monetization ▸ Pay per event. Delete the auto-added apify-default-dataset-item "Result" event (the code charges explicitly — keeping both double-charges), then add:

Event name	Title	Suggested price
keyword	Ranking keyword	$0.002 ($2 / 1,000)
competitor	Competitor found	$0.002

Both cost ~10 Semrush units, so the same price applies to each.

Revenue shape

One domain at maxKeywordsPerDomain: 5000 = 5,000 rows ≈ $10 per run — versus a user-supplied keyword list where they might paste 200. That's the reason to build this one alongside the bulk-metrics actor: it cross-sells (find competitor keywords → enrich them in bulk) and each run is worth far more.

Roadmap

1. Keyword gap — keywords competitor A ranks for that you don't.
2. Position tracking — schedule weekly runs and diff positions over time.
3. phrase_related expansion — turn top competitor keywords into thousands more.
4. URL-level analysis — group by ranking URL to find their best-performing pages.