# Competitor Domain Keywords Scraper (`automation-lab/competitor-domain-keywords-scraper`) Actor

Export current Google organic keywords for competitor domains or pages, including rank, URL, volume, CPC, competition, intent, and traffic estimates. Managed DataForSEO access; no buyer credentials required.

- **URL**: https://apify.com/automation-lab/competitor-domain-keywords-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** SEO tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Competitor Domain Keywords Scraper

Export the current Google organic keywords that a competitor domain or page ranks for. This Actor uses a managed **DataForSEO** API connection, so you do not need a DataForSEO login, subscription, proxy, or browser setup.

For every delivered keyword you get the source domain, query, organic position, exact ranking URL, search volume, nullable CPC and competition, detected intent, estimated traffic, traffic value, and keyword difficulty. Results are ready for JSON, CSV, Excel, APIs, scheduled runs, and Apify integrations.

### What can you do with competitor ranked keywords?

- discover competitor topics and landing pages;
- export a domain keyword checker dataset for content planning;
- inspect what keywords a specific competitor page ranks for;
- compare recurring exports in your warehouse or spreadsheet;
- prioritize keywords using rank, volume, intent, and traffic estimates;
- feed structured competitor research into an internal SEO workflow.

The Actor returns current organic ranking observations. It does not claim historical movement, alerts, backlinks, paid keywords, or automatic keyword-gap comparison.

### Who is this Actor for?

**SEO teams** can map a competing site's organic footprint without maintaining another API integration.

**Content strategists** can connect competitor landing pages to the queries, intent, and volume behind them.

**Agencies** can schedule repeatable exports for supplied client and competitor domains.

**Analysts and developers** can retrieve a typed dataset through the Apify API instead of normalizing provider responses themselves.

### Why use this DataForSEO alternative workflow?

This is a focused wrapper around one ranked-keyword job rather than a broad SEO suite. It provides:

- managed company credentials—buyer credentials are never accepted;
- domain and full-page URL inputs;
- four supported English-language country databases;
- organic-only, live-mode results;
- explicit nulls instead of fabricated zeroes for unavailable metrics;
- deterministic item and target limits;
- estimated and actual upstream-cost ceilings;
- clean default-dataset output and per-event billing.

The source is identified honestly: keyword and ranking metrics come from DataForSEO Labs, not direct scraping of Google pages.

### What data does the Actor extract?

| Field | Meaning |
|---|---|
| `sourceTarget` | Normalized domain or page sent to the provider |
| `sourceDomain` | Host used to group competitor rows |
| `keyword` | Google query with a current organic result |
| `searchVolume` | Estimated monthly searches, nullable |
| `cpc` | Estimated paid cost per click in USD, nullable |
| `competition` | Paid-search competition ratio, nullable |
| `competitionLevel` | Provider label such as `LOW` or `MEDIUM`, nullable |
| `intent` | Main detected search intent, nullable |
| `organicPosition` | Position among organic results, nullable |
| `absolutePosition` | Position among all SERP features, nullable |
| `rankingUrl` | Exact URL found in the result, nullable |
| `estimatedTraffic` | Estimated monthly traffic from the ranking, nullable |
| `estimatedTrafficValue` | Estimated paid replacement cost, nullable |
| `keywordDifficulty` | DataForSEO difficulty score, nullable |
| `locationCode` | Country database code |
| `languageCode` | Keyword language code |
| `checkedAt` | UTC lookup timestamp |

Nullable values remain `null`. A missing CPC is not converted to `$0`, and a missing intent is not guessed.

### How to get competitor domain keywords

1. Open the Actor input page.
2. Add one to five public domains or full page URLs.
3. Choose a supported location code.
4. Set the maximum number of keyword rows.
5. Keep the upstream-cost ceiling above the displayed workload estimate.
6. Run the Actor.
7. Download the default dataset as JSON, CSV, Excel, XML, or HTML.

A bare domain such as `dataforseo.com` requests domain-wide results. A full URL such as `https://dataforseo.com/apis/dataforseo-labs-api` requests results for that page.

### Input parameters

#### `targets`

Required array containing 1–5 unique public domains or HTTP(S) page URLs. Credentials, ports, localhost names, IP addresses, and malformed targets are rejected before paid calls.

#### `locationCode`

Country-level Google database:

- `2840` — United States (default)
- `2826` — United Kingdom
- `2124` — Canada
- `2036` — Australia

#### `languageCode`

`en` is the supported language in this release.

#### `maxItems`

Maximum rows across all targets, from 1 to 1,000. The Actor divides the limit deterministically across supplied targets. It does not silently fetch beyond the cap.

#### `maxUpstreamCostUsd`

Hard provider-cost ceiling from `$0.013` to `$0.25`, with a `$0.15` default. Before any paid request, the Actor estimates the complete workload from the documented request base and requested-row rate. It refuses workloads above the ceiling. It also totals reported response costs and stops before another target if the ceiling is exceeded.

### Example input

```json
{
  "targets": ["dataforseo.com"],
  "locationCode": "2840",
  "languageCode": "en",
  "maxItems": 10,
  "maxUpstreamCostUsd": 0.02
}
```

### Example output

This shortened row reflects current Actor output; values naturally change as Google rankings and provider metrics update.

```json
{
  "sourceTarget": "dataforseo.com",
  "sourceDomain": "dataforseo.com",
  "keyword": "dataforseo",
  "searchVolume": 2900,
  "cpc": 1.29,
  "competition": 0.52,
  "competitionLevel": "MEDIUM",
  "intent": "informational",
  "organicPosition": 1,
  "absolutePosition": 1,
  "rankingUrl": "https://dataforseo.com/",
  "estimatedTraffic": 881.6,
  "estimatedTrafficValue": 1137.26,
  "keywordDifficulty": 9,
  "locationCode": 2840,
  "languageCode": "en",
  "checkedAt": "2026-07-29T02:17:06.642Z"
}
```

### How much does it cost to extract competitor domain keywords?

Pay-per-event billing combines three charges:

- one Actor start event;
- one domain event for each successfully queried domain or page;
- one keyword event for each normalized row delivered.

At the BRONZE tier the current source configuration is `$0.001` per start, `$0.05` per successfully queried target, and `$0.0028` per delivered keyword. Examples for one target:

| Delivered rows | Example BRONZE price |
|---:|---:|
| 10 | $0.079 |
| 100 | $0.331 |
| 500 | $1.451 |

Apify plan tiers use different prices; the Console shows the active amount before you run. Empty successful target responses incur the domain event but no keyword events. Provider failures do not emit keyword rows.

The separate `maxUpstreamCostUsd` input is a safety guard for our managed provider request, not an extra amount billed to you.

### Scheduling competitor-content monitoring

Use an Apify Schedule to run the same input daily, weekly, or monthly. Each run creates a timestamped dataset. Export it to your warehouse and compare rows by `sourceDomain + keyword` to identify changes.

The Actor itself does not label a keyword as new, lost, or moved. That comparison belongs in your downstream workflow so it remains explicit and reproducible.

### Spreadsheet and database workflows

- Export CSV or Excel for one-time analysis.
- Send completed datasets to Google Sheets through an integration.
- Use a webhook to load rows into BigQuery, Snowflake, PostgreSQL, or a data lake.
- Keep `checkedAt` and run ID as snapshot keys.
- Treat nullable metrics as unknown, not zero.

### Run with the Apify API

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~competitor-domain-keywords-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"targets":["dataforseo.com"],"locationCode":"2840","languageCode":"en","maxItems":10,"maxUpstreamCostUsd":0.02}'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/competitor-domain-keywords-scraper').call({
  targets: ['dataforseo.com'],
  locationCode: '2840',
  languageCode: 'en',
  maxItems: 10,
  maxUpstreamCostUsd: 0.02,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/competitor-domain-keywords-scraper').call(run_input={
    'targets': ['dataforseo.com'],
    'locationCode': '2840',
    'languageCode': 'en',
    'maxItems': 10,
    'maxUpstreamCostUsd': 0.02,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with MCP and AI agents

Add this Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/competitor-domain-keywords-scraper"
```

#### Claude Desktop, Cursor, and VS Code setup

Use this same MCP server configuration in Claude Desktop, Cursor, or VS Code:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/competitor-domain-keywords-scraper"
    }
  }
}
```

Example prompts:

- “Export 50 US organic keywords for dataforseo.com.”
- “Find the current UK keywords for this supplied competitor landing page.”
- “Run my saved competitor-domain task and summarize high-volume informational queries.”

### Reliability and cost controls

The Actor validates all inputs before a provider request. It performs no automatic paid retry because an ambiguous retry could duplicate provider spend. HTTP errors, provider task errors, malformed response shapes, missing service configuration, and exceeded ceilings fail the run rather than returning a misleading empty dataset.

Requests use the official structured API, so no proxy or browser is needed. Memory is fixed at 256 MB, and timeout is five minutes.

### Limits

- Google organic results only;
- current live ranking mode only;
- English in the US, UK, Canada, or Australia;
- at most five targets and 1,000 requested rows per run;
- no keyword gap, backlinks, PPC-keyword export, alerts, or built-in history;
- metrics are third-party estimates and can differ from Search Console or other SEO tools;
- naturally sparse domains or pages can return fewer rows than requested.

### Legality and responsible use

Only submit domains and public page URLs you are permitted to analyze. Search metrics are estimates supplied by DataForSEO and should support—not replace—professional judgment. Follow applicable contracts, privacy rules, intellectual-property rules, and local law when storing, combining, or sharing the output.

Do not use the dataset to make deceptive claims about a site's traffic or business performance.

### Troubleshooting

#### Why was the run refused before it started?

The requested targets and row count may imply more provider cost than `maxUpstreamCostUsd` allows. Reduce `maxItems`, reduce the number of targets, or raise the ceiling within the allowed `$0.25` maximum.

#### Why did I receive fewer rows than `maxItems`?

`maxItems` is a ceiling, not a guarantee. A target may have fewer current organic keywords in the selected country, and the total limit is divided among multiple targets.

#### Why are CPC, competition, intent, or traffic fields null?

The provider did not supply that metric for the keyword. The Actor preserves nullability instead of inventing values.

#### Does the Actor need my DataForSEO login?

No. It uses company-managed credentials stored as Actor secrets. The input schema does not accept buyer credentials.

#### Can I monitor rank movement?

Schedule repeat runs and compare snapshots downstream. This release does not claim historical rank movement or emit “new/lost” labels.

### Related automation-lab Actors

- [SpyFu Domain Intelligence Scraper](https://apify.com/automation-lab/spyfu-domain-intelligence-scraper) for a separate domain-intelligence surface.
- [Semrush Website Traffic Stats Scraper](https://apify.com/automation-lab/semrush-website-traffic-stats-scraper) for traffic-oriented competitor estimates.
- [Similarweb Traffic Intelligence Scraper](https://apify.com/automation-lab/similarweb-traffic-intelligence-scraper) for website traffic and engagement signals.
- [SEO Rank Tracker](https://apify.com/automation-lab/seo-rank-tracker) when you already have a supplied keyword list to track.

### FAQ

#### Is this a Google scraper?

No browser requests are made to Google. Ranked-keyword records come from the official DataForSEO Labs API.

#### Are results real time?

The Actor requests the provider's current live-mode database. It does not promise instant SERP observations or exact Search Console values.

#### Can I analyze one page instead of a whole domain?

Yes. Supply a complete `https://` page URL. A bare host requests domain-wide results.

#### Can I provide more than five competitors?

Split larger lists into separate runs or Tasks. The five-target cap keeps spend and runtime bounded.

#### What happens when the provider fails?

The run fails with a concise error and no automatic paid retry. Already delivered rows remain in the run dataset only if an earlier target completed successfully.

# Actor input Schema

## `targets` (type: `array`):

One to five public domains (for example, dataforseo.com) or full page URLs. Domain inputs return domain-wide ranked keywords; full page URLs return keywords for that page.

## `locationCode` (type: `string`):

Country-level Google database used for ranking and keyword metrics.

## `languageCode` (type: `string`):

Keyword language. English is supported in this release.

## `maxItems` (type: `integer`):

Maximum rows across all targets. The limit is divided deterministically among targets and controls upstream cost before requests start.

## `maxUpstreamCostUsd` (type: `number`):

Hard DataForSEO cost ceiling for this run. The Actor refuses the run before paid calls if estimated cost exceeds it, and stops if reported actual cost exceeds it.

## Actor input object example

```json
{
  "targets": [
    "dataforseo.com"
  ],
  "locationCode": "2840",
  "languageCode": "en",
  "maxItems": 20,
  "maxUpstreamCostUsd": 0.15
}
```

# Actor output Schema

## `overview` (type: `string`):

Open current keyword, position, URL, volume, CPC, intent, traffic, and source-domain rows in the default dataset overview.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "dataforseo.com"
    ],
    "locationCode": "2840",
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/competitor-domain-keywords-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "targets": ["dataforseo.com"],
    "locationCode": "2840",
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/competitor-domain-keywords-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "dataforseo.com"
  ],
  "locationCode": "2840",
  "maxItems": 20
}' |
apify call automation-lab/competitor-domain-keywords-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=automation-lab/competitor-domain-keywords-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/uJZiTzIkBMtDF9dc9/builds/ZRlg3ZEprPknyn2lg/openapi.json