SERP Multi — 6 Search Engines × Country-Native Routing

Aggregated organic search-results scraper for Google, Bing, DuckDuckGo, Yahoo, Baidu, and Yandex, with built-in country-native domain & language routing across 25+ markets. One JSON input → one structured dataset row per organic result → one cross-engine SUMMARY.json.

A multi-engine alternative to SerpAPI: you don't lose Baidu's Chinese-language coverage or Yandex's Russian results when you change country, and you can compare cross-engine ranking in a single run.

Quick start

Open the Actor in Apify Console, paste the input below, run.

{
  "queries": ["coffee maker", "best wireless earbuds"],
  "country": "US",
  "engines": ["google", "bing", "duckduckgo", "yahoo", "baidu", "yandex"],
  "resultsPerQuery": 10,
  "safeSearch": "moderate",
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Input

Field	Required	Description
queries	yes	1–50 search queries (any language)
country	yes	ISO 3166-1 alpha-2 — picked from the supported 25-country dropdown
engines	no	Subset of {google, bing, duckduckgo, yahoo, baidu, yandex}. Leave empty/all-selected = query every supported engine. Baidu/Yandex auto-skip outside their native markets unless explicitly selected
resultsPerQuery	no	1–100, default 10. Note: Google often returns fewer than resultsPerQuery organic rows because it reserves slots for People Also Ask, news, videos, etc. Set 15–20 if you want 10 organic
language	no	ISO 639-1 override (e.g. en, zh, ru). Default derived from country
safeSearch	no	off / moderate / strict; default moderate
maxRequestsPerRun	no	default 5000
maxConcurrency	no	default 10
summaryTopN	no	top-N domains per query in SUMMARY; default 50 (0 disables)
proxyConfiguration	no	default { useApifyProxy: true, apifyProxyGroups: ["RESIDENTIAL"] }. Residential strongly recommended

Output

Dataset — one row per organic result

{
  "query": "coffee maker",
  "engine": "google",
  "position": 1,
  "title": "Best Coffee Makers 2026 — Wirecutter",
  "url": "https://www.nytimes.com/wirecutter/...",
  "displayUrl": "nytimes.com › wirecutter › ...",
  "snippet": "Our pick after 80 hours of testing is the …",
  "country": "US",
  "language": "en",
  "scrapedAt": "2026-05-13T05:01:43.502Z"
}

Key-value store SUMMARY — cross-engine analytics

• meta — country, language, queries, enginesUsed/Skipped/WithErrors, per-engine attempt/success/blocked/empty counts, request timings, Crawlee stats
• perQuery[query]:
  • totalResults, uniqueDomains
  • topDomainsByCoverage — domains ranked by how many engines surface them, with best position
  • overlapMatrix — Jaccard similarity between every engine pair on this query's URL set (host+path, query-string stripped)

Example: overlapMatrix["bing-vs-google"] = 0.55 means 55% of the URLs are common between Bing and Google for that query.

Engine notes

Google (delegated)

Google is delegated to Apify's official apify/google-search-scraper Actor via Actor.call(). They handle JavaScript rendering, CAPTCHA solving, IP rotation — work we can't replicate without specialized infrastructure. You'll see a child Actor run in your Apify Console for every serp-multi run that includes Google, and your bill will include that child Actor's cost (priced per result by the upstream Actor).

If you want to skip Google to avoid the child-Actor cost, set engines to anything excluding "google".

Bing, DuckDuckGo, Yahoo, Baidu, Yandex (in-house)

These five engines are scraped directly via Cheerio + Apify residential proxies. Each engine has its own adapter in src/adapters/ that builds the right URL and parses the SERP HTML. Block-detection per engine is reported in SUMMARY.meta.enginesWithErrors.

Country routing & soft-skip rules

Each country has a native-engines list in src/locale.js. By default:

• Baidu only runs for CN (and where you've explicitly selected it).
• Yandex only runs for RU/TR (and where you've explicitly selected it).
• Yahoo is hard-skipped for countries with no working Yahoo subdomain (e.g., SE, PL, KR, CN, RU…).

You can force any engine for any country by explicitly listing it in engines. A warning is logged if results are likely to be weak.

Pricing & cost model

Two cost components:

1. serp-multi compute — small; mostly idle except during result aggregation.
2. apify/google-search-scraper compute + per-result fee — only when Google is included. See the upstream Actor's pricing.

To estimate: each query × Google ≈ 1 child-Actor task. Multiple queries are batched into a single Actor.call, so 10 queries = 1 child run.

Local development

npm install
npm test                 # 39 unit tests (locale, adapter parsers, aggregate)
npm run start            # local Actor run (reads .actor/INPUT.json)
npm run test:integration # manual real-network smoke (does NOT delegate Google)

Note: a local npm run start will try Actor.call('apify/google-search-scraper', ...) only if your local environment has a valid Apify token (APIFY_TOKEN env var); otherwise the Google engine is reported as failed and other engines still run.

Architecture

• src/main.js — orchestration: validate → resolve locale → filter engines → split (Google → Apify Actor.call, others → CheerioCrawler) → aggregate → SUMMARY → exit
• src/adapters/ — one file per engine. Each exports { name, buildUrl(...), parseResponse(html, $) }. Google's adapter is retained for unit-test value and a future self-hosted fallback path.
• src/locale.js — country → per-engine routing params (Google domain/gl/hl, Bing cc/setlang, etc.). 25 countries + generic English fallback.
• src/aggregate.js — dataset rows → SUMMARY (overlapMatrix + topDomainsByCoverage + error thresholds).
• test/ — node:test + fixtures captured from real engine HTML.

Limitations

• v0.3 returns organic results only — no ads, knowledge panel, featured snippet, related searches, images, videos, news, or shopping. Those modules are deferred to v0.4+.
• One country per run (cross-country comparison requires multiple runs).
• Pagination is implicit (we request num=100 / count=50 / engine-equivalent in a single HTTP request); no start=10/20/… paging support.
• Yandex reliability: Yandex aggressively serves a CAPTCHA verification page (<title>Верификация</title>) to scraper traffic. Even with apifyProxyCountry: 'RU' + session rotation, individual queries may exhaust all retries with CAPTCHA. Expect 30-70% per-query success rate on RU country; rerun if needed. Bing/DuckDuckGo/Google work reliably on RU.
• Yahoo: A/B layout variants occasionally produce 0-row results even on supported countries. Multiple selector fallbacks are in place but the failure mode isn't fully eliminated.
• DuckDuckGo is auto-skipped for CN (network-level blocked); SUMMARY's enginesSkipped will indicate the reason.