Competitor Domain Keywords Scraper

Export the current Google organic keywords that a competitor domain or page ranks for. This Actor uses a managed DataForSEO API connection, so you do not need a DataForSEO login, subscription, proxy, or browser setup.

For every delivered keyword you get the source domain, query, organic position, exact ranking URL, search volume, nullable CPC and competition, detected intent, estimated traffic, traffic value, and keyword difficulty. Results are ready for JSON, CSV, Excel, APIs, scheduled runs, and Apify integrations.

What can you do with competitor ranked keywords?

• discover competitor topics and landing pages;
• export a domain keyword checker dataset for content planning;
• inspect what keywords a specific competitor page ranks for;
• compare recurring exports in your warehouse or spreadsheet;
• prioritize keywords using rank, volume, intent, and traffic estimates;
• feed structured competitor research into an internal SEO workflow.

The Actor returns current organic ranking observations. It does not claim historical movement, alerts, backlinks, paid keywords, or automatic keyword-gap comparison.

Who is this Actor for?

SEO teams can map a competing site's organic footprint without maintaining another API integration.

Content strategists can connect competitor landing pages to the queries, intent, and volume behind them.

Agencies can schedule repeatable exports for supplied client and competitor domains.

Analysts and developers can retrieve a typed dataset through the Apify API instead of normalizing provider responses themselves.

Why use this DataForSEO alternative workflow?

This is a focused wrapper around one ranked-keyword job rather than a broad SEO suite. It provides:

• managed company credentials—buyer credentials are never accepted;
• domain and full-page URL inputs;
• four supported English-language country databases;
• organic-only, live-mode results;
• explicit nulls instead of fabricated zeroes for unavailable metrics;
• deterministic item and target limits;
• estimated and actual upstream-cost ceilings;
• clean default-dataset output and per-event billing.

The source is identified honestly: keyword and ranking metrics come from DataForSEO Labs, not direct scraping of Google pages.

What data does the Actor extract?

Field	Meaning
sourceTarget	Normalized domain or page sent to the provider
sourceDomain	Host used to group competitor rows
keyword	Google query with a current organic result
searchVolume	Estimated monthly searches, nullable
cpc	Estimated paid cost per click in USD, nullable
competition	Paid-search competition ratio, nullable
competitionLevel	Provider label such as LOW or MEDIUM, nullable
intent	Main detected search intent, nullable
organicPosition	Position among organic results, nullable
absolutePosition	Position among all SERP features, nullable
rankingUrl	Exact URL found in the result, nullable
estimatedTraffic	Estimated monthly traffic from the ranking, nullable
estimatedTrafficValue	Estimated paid replacement cost, nullable
keywordDifficulty	DataForSEO difficulty score, nullable
locationCode	Country database code
languageCode	Keyword language code
checkedAt	UTC lookup timestamp

Nullable values remain null. A missing CPC is not converted to $0, and a missing intent is not guessed.

How to get competitor domain keywords

1. Open the Actor input page.
2. Add one to five public domains or full page URLs.
3. Choose a supported location code.
4. Set the maximum number of keyword rows.
5. Keep the upstream-cost ceiling above the displayed workload estimate.
6. Run the Actor.
7. Download the default dataset as JSON, CSV, Excel, XML, or HTML.

A bare domain such as dataforseo.com requests domain-wide results. A full URL such as https://dataforseo.com/apis/dataforseo-labs-api requests results for that page.

Input parameters

targets

Required array containing 1–5 unique public domains or HTTP(S) page URLs. Credentials, ports, localhost names, IP addresses, and malformed targets are rejected before paid calls.

locationCode

Country-level Google database:

• 2840 — United States (default)
• 2826 — United Kingdom
• 2124 — Canada
• 2036 — Australia

languageCode

en is the supported language in this release.

maxItems

Maximum rows across all targets, from 1 to 1,000. The Actor divides the limit deterministically across supplied targets. It does not silently fetch beyond the cap.

maxUpstreamCostUsd

Hard provider-cost ceiling from $0.013 to $0.25, with a $0.15 default. Before any paid request, the Actor estimates the complete workload from the documented request base and requested-row rate. It refuses workloads above the ceiling. It also totals reported response costs and stops before another target if the ceiling is exceeded.

Example input

{
  "targets": ["dataforseo.com"],
  "locationCode": "2840",
  "languageCode": "en",
  "maxItems": 10,
  "maxUpstreamCostUsd": 0.02
}

Example output

This shortened row reflects current Actor output; values naturally change as Google rankings and provider metrics update.

{
  "sourceTarget": "dataforseo.com",
  "sourceDomain": "dataforseo.com",
  "keyword": "dataforseo",
  "searchVolume": 2900,
  "cpc": 1.29,
  "competition": 0.52,
  "competitionLevel": "MEDIUM",
  "intent": "informational",
  "organicPosition": 1,
  "absolutePosition": 1,
  "rankingUrl": "https://dataforseo.com/",
  "estimatedTraffic": 881.6,
  "estimatedTrafficValue": 1137.26,
  "keywordDifficulty": 9,
  "locationCode": 2840,
  "languageCode": "en",
  "checkedAt": "2026-07-29T02:17:06.642Z"
}

How much does it cost to extract competitor domain keywords?

Pay-per-event billing combines three charges:

• one Actor start event;
• one domain event for each successfully queried domain or page;
• one keyword event for each normalized row delivered.

At the BRONZE tier the current source configuration is $0.001 per start, $0.05 per successfully queried target, and $0.0028 per delivered keyword. Examples for one target:

Delivered rows	Example BRONZE price
10	$0.079
100	$0.331
500	$1.451

Apify plan tiers use different prices; the Console shows the active amount before you run. Empty successful target responses incur the domain event but no keyword events. Provider failures do not emit keyword rows.

The separate maxUpstreamCostUsd input is a safety guard for our managed provider request, not an extra amount billed to you.

Scheduling competitor-content monitoring

Use an Apify Schedule to run the same input daily, weekly, or monthly. Each run creates a timestamped dataset. Export it to your warehouse and compare rows by sourceDomain + keyword to identify changes.

The Actor itself does not label a keyword as new, lost, or moved. That comparison belongs in your downstream workflow so it remains explicit and reproducible.

Spreadsheet and database workflows

• Export CSV or Excel for one-time analysis.
• Send completed datasets to Google Sheets through an integration.
• Use a webhook to load rows into BigQuery, Snowflake, PostgreSQL, or a data lake.
• Keep checkedAt and run ID as snapshot keys.
• Treat nullable metrics as unknown, not zero.

Run with the Apify API

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~competitor-domain-keywords-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"targets":["dataforseo.com"],"locationCode":"2840","languageCode":"en","maxItems":10,"maxUpstreamCostUsd":0.02}'

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/competitor-domain-keywords-scraper').call({
  targets: ['dataforseo.com'],
  locationCode: '2840',
  languageCode: 'en',
  maxItems: 10,
  maxUpstreamCostUsd: 0.02,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/competitor-domain-keywords-scraper').call(run_input={
    'targets': ['dataforseo.com'],
    'locationCode': '2840',
    'languageCode': 'en',
    'maxItems': 10,
    'maxUpstreamCostUsd': 0.02,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use with MCP and AI agents

Add this Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/competitor-domain-keywords-scraper"

Claude Desktop, Cursor, and VS Code setup

Use this same MCP server configuration in Claude Desktop, Cursor, or VS Code:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/competitor-domain-keywords-scraper"
    }
  }
}

Example prompts:

• “Export 50 US organic keywords for dataforseo.com.”
• “Find the current UK keywords for this supplied competitor landing page.”
• “Run my saved competitor-domain task and summarize high-volume informational queries.”

Reliability and cost controls

The Actor validates all inputs before a provider request. It performs no automatic paid retry because an ambiguous retry could duplicate provider spend. HTTP errors, provider task errors, malformed response shapes, missing service configuration, and exceeded ceilings fail the run rather than returning a misleading empty dataset.

Requests use the official structured API, so no proxy or browser is needed. Memory is fixed at 256 MB, and timeout is five minutes.

Limits

• Google organic results only;
• current live ranking mode only;
• English in the US, UK, Canada, or Australia;
• at most five targets and 1,000 requested rows per run;
• no keyword gap, backlinks, PPC-keyword export, alerts, or built-in history;
• metrics are third-party estimates and can differ from Search Console or other SEO tools;
• naturally sparse domains or pages can return fewer rows than requested.

Legality and responsible use

Only submit domains and public page URLs you are permitted to analyze. Search metrics are estimates supplied by DataForSEO and should support—not replace—professional judgment. Follow applicable contracts, privacy rules, intellectual-property rules, and local law when storing, combining, or sharing the output.

Do not use the dataset to make deceptive claims about a site's traffic or business performance.

Troubleshooting

Why was the run refused before it started?

The requested targets and row count may imply more provider cost than maxUpstreamCostUsd allows. Reduce maxItems, reduce the number of targets, or raise the ceiling within the allowed $0.25 maximum.

Why did I receive fewer rows than maxItems?

maxItems is a ceiling, not a guarantee. A target may have fewer current organic keywords in the selected country, and the total limit is divided among multiple targets.

Why are CPC, competition, intent, or traffic fields null?

The provider did not supply that metric for the keyword. The Actor preserves nullability instead of inventing values.

Does the Actor need my DataForSEO login?

No. It uses company-managed credentials stored as Actor secrets. The input schema does not accept buyer credentials.

Can I monitor rank movement?

Schedule repeat runs and compare snapshots downstream. This release does not claim historical rank movement or emit “new/lost” labels.

Related automation-lab Actors

• SpyFu Domain Intelligence Scraper for a separate domain-intelligence surface.
• Semrush Website Traffic Stats Scraper for traffic-oriented competitor estimates.
• Similarweb Traffic Intelligence Scraper for website traffic and engagement signals.
• SEO Rank Tracker when you already have a supplied keyword list to track.

FAQ

Is this a Google scraper?

No browser requests are made to Google. Ranked-keyword records come from the official DataForSEO Labs API.

Are results real time?

The Actor requests the provider's current live-mode database. It does not promise instant SERP observations or exact Search Console values.

Can I analyze one page instead of a whole domain?

Yes. Supply a complete https:// page URL. A bare host requests domain-wide results.

Can I provide more than five competitors?

Split larger lists into separate runs or Tasks. The five-target cap keeps spend and runtime bounded.

What happens when the provider fails?

The run fails with a concise error and no automatic paid retry. Already delivered rows remain in the run dataset only if an earlier target completed successfully.