SpyFu Domain Intelligence Scraper

Extract public SpyFu domain intelligence at scale for SEO and PPC competitor research. Add a list of domains and get structured JSON rows with domain validation, country keyword counts, SEO overview estimates, organic movement summaries, and sample organic keywords.

What does SpyFu Domain Intelligence Scraper do?

SpyFu Domain Intelligence Scraper turns SpyFu's public domain research views into a clean Apify dataset. It is designed for teams that need repeatable competitor checks rather than manual copy-paste from the SpyFu interface.

The actor normalizes domains, checks whether SpyFu recognizes each domain, fetches country-level keyword counts, retrieves public SEO summary metrics, and optionally includes a small sample of most valuable organic keywords.

Who is it for? SpyFu SEO and PPC teams

• 🧑‍💼 SEO agencies auditing client competitors in bulk
• 📈 PPC teams triaging competitor domains before deeper ad research
• 🧪 Growth marketers building competitive-intelligence spreadsheets
• 🏢 B2B lead researchers qualifying company domains by search footprint
• 🧾 Consultants preparing recurring monthly SEO/PPC reports

Why use this actor?

Manual SpyFu checks are slow when you have dozens of domains. This actor gives you a reproducible Apify run, downloadable JSON/CSV/Excel outputs, API access, and dataset history for recurring workflows.

Data you can extract

Field	Description
domain	Normalized domain submitted to the actor
queryFound	Whether SpyFu recognized the domain query
cleansedDomain	SpyFu-normalized domain
queryId	SpyFu query identifier when visible
countryKeywordStats	Keyword counts by available SpyFu country database
organicSummary	Public SEO overview metrics when visible
organicMovementStats	New/gained/lost organic movement summaries
organicKeywords	Optional sample of public most valuable organic keywords
sourceUrl	SpyFu page URL for manual follow-up
error	Per-domain error details when a section is unavailable

How much does it cost to scrape SpyFu domain intelligence?

This actor uses pay-per-event pricing. There is a small start event and one domain event per dataset row. You only pay for domains that are processed and saved to the dataset.

Run size	Charged events	Estimated FREE-tier price*	Typical use case
3 domains	1 start + 3 domain events	~$0.0066	Default smoke test
25 domains	1 start + 25 domain events	~$0.0107	Small competitor shortlist
50 domains	1 start + 50 domain events	~$0.0154	Agency prospect batch
100 domains	1 start + 100 domain events	~$0.0247	Realistic bulk enrichment run
1,000 domains	1 start + 1,000 domain events	~$0.1933	Large recurring SEO/PPC report

*Estimates use the current FREE-tier event prices configured for this actor: $0.006 per run start and $0.00018731 per processed domain. Higher Apify plan tiers receive lower per-domain event prices where available. Your final Apify bill can also include normal platform compute/proxy charges shown by Apify for your account.

With a $5 free Apify credit budget, the domain-event portion supports roughly 26,000 processed domains before compute/proxy overhead. A more conservative planning estimate is about 200 realistic 100-domain runs at the event-price level, so first-time users can test several small batches before scaling.

The default input is intentionally small so first runs are cheap. For large lists, start with a short sample, review output quality, and then scale up.

How to use

1. Open the actor on Apify.
2. Add domains such as apify.com, semrush.com, or example.com.
3. Keep US as the country database or choose another supported country.
4. Choose whether to include organic keyword samples.
5. Run the actor.
6. Download results from the dataset as JSON, CSV, Excel, or via API.

Input example

{
  "domains": ["apify.com", "semrush.com", "spyfu.com"],
  "countryCode": "US",
  "includeOrganicKeywords": true,
  "maxKeywordsPerDomain": 5,
  "requestDelayMs": 750
}

Output example

{
  "domain": "apify.com",
  "queryFound": true,
  "cleansedDomain": "apify.com",
  "countryCode": "US",
  "primaryCountry": "US",
  "countryKeywordStats": [{ "countryCode": "US", "keywords": 95130, "isPrimary": true }],
  "organicKeywords": [{ "keyword": "web scraper", "rank": 12 }],
  "sourceUrl": "https://www.spyfu.com/overview/domain?query=apify.com",
  "scrapedAt": "2026-07-01T00:00:00.000Z"
}

Input fields

Domains

A required list of domains or URLs. URLs are normalized to hostnames and www. is removed.

Country code

SpyFu country database code. US is the default and usually the richest public dataset.

Include organic keyword samples

Enable this to include a bounded keyword sample for each domain. Disable it for faster domain validation runs.

Max organic keywords per domain

Controls the size of the nested organicKeywords array. Keep it low for quick competitive triage.

Request delay

Adds a delay between domains to reduce pressure on SpyFu public endpoints.

Tips for best results

• Use root domains rather than long URLs when possible.
• Start with 5-10 domains before scaling to a large list.
• Keep keyword samples small for monitoring workflows.
• Use the sourceUrl field when a domain needs manual follow-up in SpyFu.
• Review the error field for domains where one public section was unavailable.

Integrations

Use this actor with:

• Google Sheets exports for agency reporting
• Apify webhooks for recurring competitor monitoring
• CRM enrichment flows where domains need an SEO footprint signal
• Data warehouses that combine SpyFu output with Search Console or Ads data
• BI dashboards that track competitor movement over time

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/spyfu-domain-intelligence-scraper').call({
  domains: ['apify.com', 'spyfu.com'],
  countryCode: 'US'
});
console.log(run.defaultDatasetId);

Python

from apify_client import ApifyClient

client = ApifyClient('MY-APIFY-TOKEN')
run = client.actor('automation-lab/spyfu-domain-intelligence-scraper').call(run_input={
    'domains': ['apify.com', 'spyfu.com'],
    'countryCode': 'US',
})
print(run['defaultDatasetId'])

cURL

curl -X POST 'https://api.apify.com/v2/acts/automation-lab~spyfu-domain-intelligence-scraper/runs?token=MY-APIFY-TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"domains":["apify.com","spyfu.com"],"countryCode":"US"}'

MCP usage

Connect Apify MCP to Claude Desktop or Claude Code and enable this actor with:

$claude mcp add apify-spyfu https://mcp.apify.com/?tools=automation-lab/spyfu-domain-intelligence-scraper

Claude Desktop JSON configuration example:

{
  "mcpServers": {
    "apify-spyfu": {
      "url": "https://mcp.apify.com/?tools=automation-lab/spyfu-domain-intelligence-scraper"
    }
  }
}

Example prompts:

• "Run the SpyFu Domain Intelligence Scraper for these 20 competitor domains and summarize primary countries."
• "Compare organic keyword samples for apify.com and spyfu.com."
• "Export SpyFu domain intelligence for my CRM enrichment list."

Reliability notes

SpyFu's browser app may throttle automated browser sessions. This actor uses public JSON endpoints and conservative delays instead of rendering the full browser app. If a section is unavailable, the actor saves a row with an error field rather than failing the whole run.

Legality and ethical use

This actor extracts data visible from public SpyFu endpoints. Use it responsibly, obey applicable laws, respect SpyFu's terms, and avoid excessive request rates. Do not use the actor to collect personal data or bypass access controls.

FAQ

Does this require a SpyFu account?

No. The actor is built for public, unauthenticated SpyFu domain intelligence endpoints.

Why are PPC ads or backlinks not always included?

Some high-value sections are not reliably exposed through public unauthenticated endpoints. The actor focuses on sections it can retrieve consistently without login.

Why do some domains have empty keyword samples?

SpyFu may not have public data for the domain/country pair, or a public endpoint may return only aggregate stats.

What should I do if I see errors?

Increase requestDelayMs, reduce keyword samples, and rerun a small domain subset. Per-domain errors are saved in the dataset for debugging.

Related scrapers

Explore other Automation Lab actors at:

• https://apify.com/automation-lab/google-search-scraper
• https://apify.com/automation-lab/google-ads-transparency-scraper
• https://apify.com/automation-lab/website-contact-finder

Changelog

• 0.1 — Initial public SpyFu domain intelligence scraper with domain validation, country keyword stats, SEO summaries, and keyword samples.

Support

If a run fails or a field looks wrong, share the run URL and a short description of the expected domain data. Include the input domains and country code so the issue can be reproduced.