Bulk Keyword Metrics (Semrush) — No 100 Limit

Look up search volume, CPC, competition and trend (and optionally keyword difficulty) for unlimited keywords in one run — past Semrush's 100-keyword cap. Paste thousands, get one clean, deduped file.

How it works

Semrush's Analytics API caps a batch at 100 phrases per request. This actor just orchestrates around it:

N keywords → dedup → chunk into 100s → phrase_these per batch
                                        (+ phrase_kdi if difficulty enabled)
                                        → merge → dataset  (charge per keyword)

• phrase_these → volume, CPC, competition, results, trend (~10 Semrush units/keyword).
• phrase_kdi → keyword difficulty (~50 units/keyword — 5× more, so it's optional).

Cost model — read this before pricing 💰

You embed your Semrush API key, so every run spends your units. Make it self-funding: price the pay-per-event above your unit cost + Apify's revenue share.

Metric	Semrush report	~units / keyword	PPE event
Volume + CPC + competition + trend	phrase_these	~10	keyword
Keyword difficulty	phrase_kdi	~50	keyword_kd

So a difficulty run costs ~6× a plain run — which is why difficulty is a separate, opt-in event (keyword_kd), charged only for keywords that return a KD value. Set each event's price above the units it consumes and every run profits. Confirm your real per-line unit cost on your plan before publishing (see "Verify" below) — that number is your price floor.

Safety rails (protect your Semrush balance)

• Dedup (case-insensitive) — never pay twice for the same keyword.
• maxKeywords hard cap — bounds any single run's cost.
• Units pre-flight — estimates units needed and warns if your balance is lower.
• Fatal-error stop — bad key / quota / disabled-report errors halt the run instead of looping and burning units; you still keep (and charge for) rows already delivered.

Setup

• Operator (embedded key): set SEMRUSH_API_KEY as an environment variable on the actor (Apify Console → Actor → Settings → Environment variables, mark secret).
• Bring-your-own-key mode: leave the env unset and let users pass semrushApiKey.

Run locally

cd semrush-bulk
npm install
# PowerShell:
$env:SEMRUSH_API_KEY = "your-semrush-api-key"
# edit storage/key_value_stores/default/INPUT.json, then:
node src/main.js

Example storage/key_value_stores/default/INPUT.json:

{
  "keywords": ["seo tools", "keyword research", "backlink checker"],
  "databases": ["us", "uk"],
  "includeKeywordDifficulty": false,
  "maxKeywords": 10000
}

Input

Field	Meaning
keywords	list of keywords (no 100 limit)
keywordsText	bulk paste (newline/comma separated), merged with keywords
databases	country markets, e.g. ["us","uk","de"] — each multiplies cost
includeKeywordDifficulty	also fetch KD (5× cost)
maxKeywords	hard cap per run
xlsxOutput	also save a formatted keywords.xlsx (default on)
semrushApiKey	optional key override (else SEMRUSH_API_KEY env)

Output (one row per keyword × database)

{
  "keyword": "seo tools",
  "database": "us",
  "volume": 12100,
  "cpc": 3.5,
  "competition": 0.42,
  "results": 1000000,
  "trend": "0.80,0.90,1.00",
  "difficulty": 78
}

Excel export

With xlsxOutput on (default), the run also writes a formatted keywords.xlsx to the key-value store — bold/filterable header, frozen top row, and number formats (volume, $ CPC, competition, difficulty). Download it from the run's Storage → Key-value store tab, or via the logged URL. The dataset (JSON/CSV) is still produced as usual.

Publish + pay-per-event

apify login
apify push

Then Apify Console ▸ your actor ▸ Publication ▸ Monetization ▸ Pay per event and add:

1. Event keyword — priced above ~10 units of cost (the base metric).
2. Event keyword_kd — priced above ~50 units of cost (only when difficulty is on).

The code charges per delivered row, so users pay for data they actually get.

Verify your real unit cost (do this first)

The unit numbers above are approximate — your plan's real per-line cost sets your price floor. Put your key in .env (SEMRUSH_API_KEY=..., gitignored) and check your balance before/after a tiny run:

$curl "https://www.semrush.com/users/countapiunits.html?key=YOUR_KEY"   # balance

Run 5 keywords, check the balance again, and divide — that's your true cost per keyword.

Roadmap

1. Intent tagging — classify each keyword (informational/commercial/transactional) with a cheap LLM pass for a value-add competitors don't have.
2. More columns — SERP features, results, via extra export_columns.
3. Excel export — agency-friendly .xlsx output.
4. Related/broad-match expansion — turn seeds into thousands via phrase_related.