Bing, DuckDuckGo, Baidu & Yahoo SERP Scraper

Scrape organic search results from the four engines Google-only scrapers ignore — Bing, DuckDuckGo, Baidu and Yahoo — in a single run. Pass your queries, pick engines, get clean structured JSON: title, URL, snippet, rank, page, engine tag and the engine's estimated total-results count. Language/region targeting, native pagination, optional desktop/mobile device targeting. No API keys, no login, no browser.

Why not Google? Because Google is already a solved, commoditised market — Apify's own Google Search Scraper does it well and cheaply, and a dozen others race it to the bottom. Meanwhile Bing (~4% of desktop search and the engine behind Copilot), DuckDuckGo, Baidu (China) and Yahoo (Japan) are barely covered, and a Bing-only scraper on this store will charge you more than this Actor charges for all four. If your rank tracking stops at Google, you are blind on every engine your competitors are not.

Engine parsers ported from the MIT-licensed openserp project (see NOTICE).

What SERP data does this scraper extract?

Each result is one flat JSON record per organic (or ad, if enabled) result:

Field	Meaning
query	The search query this result came from
engine	Engine that returned it: bing, duckduckgo, baidu, yahoo
rank	1-based organic rank within (query, engine), across pages
page	1-based SERP page number
title	Result title as shown on the SERP
url	Target URL (redirect wrappers unwrapped where possible)
snippet	Description / snippet text
ad	true when the result is a sponsored/ad slot
serpUrl	The exact SERP URL that was fetched (for reproducibility)
device	Device profile used for the request: desktop or mobile (echoes the device input; see Device targeting below)
resultsTotal	The engine's own estimated total number of matches for the query (e.g. Bing's "About 1,230,000 results"). Best-effort — populated for bing and baidu when the SERP shows a count header; null for duckduckgo / yahoo (no count header) and whenever the header wasn't present

Every field above is present on every row (flat schema, no nested objects, no permanently-empty columns) — this keeps CSV/Excel exports clean.

How to scrape search engine results with this Actor

1. Click Try for free / Run — no login to any search engine, no cookies.
2. Enter one or more queries, tick the engines you want, optionally set languageCode / region / maxPagesPerQuery.
3. Run it and export the dataset as JSON, CSV or Excel, or read it over the API.

Run it from your own code:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("nomad-agent/serp-multi-engine-scraper").call(run_input={
    "queries": ["best python web framework"],
    "engines": ["bing", "duckduckgo", "baidu"],
    "maxPagesPerQuery": 2,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["engine"], item["rank"], item["title"], item["url"])

Or a single HTTP call that runs the Actor and returns items in one response:

curl -X POST \
  "https://api.apify.com/v2/acts/nomad-agent~serp-multi-engine-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["best python web framework"], "engines": ["bing", "duckduckgo"]}'

Integrations: export the dataset as JSON, CSV or XLSX from the Console; call it synchronously with run-sync-get-dataset-items as shown above; wire it into Make, Zapier or n8n via their Apify connector; or drive it from any AI agent through the Apify MCP server.

Input

Field	Type	Default	Notes
queries	array	—	Search queries, one per entry. Inline operators like site: and filetype: work. Required.
engines	array	["bing","duckduckgo"]	Any of bing, duckduckgo, baidu, yahoo.
device	string	"desktop"	desktop or mobile. Only changes behavior on Bing — see Device targeting. Other engines accept and ignore it.
languageCode	string	""	BCP-47 code (en, de, pt-BR). Mapped per engine: Bing mkt/setlang/cc, DuckDuckGo kl, Yahoo vl. Baidu takes no language param.
region	string	""	Country hint: two-letter code for Bing (cc), e.g. de, gb. Ignored by engines without a region param.
maxPagesPerQuery	integer	1	SERP pages per query per engine, via each engine's native offset param. Stops early on empty/blocked pages. Max 10.
resultsPerPage	integer	0	Results per page where supported (Bing count, Baidu rn, Yahoo n). 0 = engine default. Max 50.
maxItems	integer	100	Hard cap on total results pushed across all queries and engines, shared evenly across your queries so none is starved. Each result returned is billed as one dataset item — see Pricing below. 0 = no cap (costs then scale with queries × engines × pages — use with care). Max 1000.
includeAds	boolean	false	Include sponsored results (tagged "ad": true).
proxyConfiguration	object	off	Optional. Yahoo needs Apify RESIDENTIAL proxy; Bing, DuckDuckGo and Baidu work without one.
concurrency	integer	4	(Advanced) How many searches run at the same time. Higher is faster but more likely to get rate-limited. Extra pages within one search are always fetched one at a time with a short delay, regardless of this setting. Max 8.

Output example

Real push_data shape, captured from a live run:

{
  "query": "open source search engine",
  "engine": "duckduckgo",
  "rank": 1,
  "page": 1,
  "title": "Home - OpenSearch",
  "url": "https://opensearch.org/",
  "snippet": "OpenSearch is an open-source search and analytics suite…",
  "ad": false,
  "serpUrl": "https://html.duckduckgo.com/html/?q=open+source+search+engine&kl=us-en",
  "device": "desktop",
  "resultsTotal": null
}

resultsTotal is null on DuckDuckGo and Yahoo — neither renders a total-match count on the endpoints this Actor reads.

Pricing

Pay per event — no subscription, no rental, you pay only for what you fetch.

	Actor start	Per result	1,000 results
From 20 July 2026	$0.005	$0.001	$1.00
Until 19 July 2026	$0.05	$0.004	$4.05

A price cut is already scheduled and takes effect on 20 July 2026 (Apify requires 14 days' notice for any price change, so it cannot land sooner). Until then the old rate above applies — quoted here in full rather than advertising a price you would not actually be charged.

maxItems (default 100) is a hard ceiling on what you can be billed for in a run.

Use cases

• Rank tracking / SEO monitoring on the engines Google-only tools miss
• Bing rank tracking — the engine behind Microsoft Copilot's web answers
• Baidu (China) and Yahoo (Japan) SERP snapshots for regional SEO
• Comparing result overlap between Bing, DuckDuckGo, Baidu and Yahoo
• Regional / language SERP snapshots (languageCode + region)
• Feeding organic results into research, lead-gen or LLM pipelines
• Cheap keyword-to-URL discovery without a paid SERP API

Engines & reliability

Engine	Proxy needed?	Notes
Bing	No	HTML first, automatic RSS-output fallback when Bing serves its JS bot-wall
DuckDuckGo	No	Uses the JS-free html.duckduckgo.com endpoint
Baidu	No	Real target URLs taken from the result mu attribute where present
Yahoo	Yes — residential	Returns nothing from Apify's datacenter IPs. Enable Apify Proxy (RESIDENTIAL) and it works. r.search.yahoo.com redirect wrappers are unwrapped to the real target URL

Every page that comes back blocked or empty is retried once on a fresh session before the Actor gives up on it. Engines that stay blocked are logged and skipped — they never fail the run, and you are never charged for results that were not returned.

Why Google, Yandex and Ecosia are not offered

They were, and they never worked. All three captcha-wall a plain HTTP request — verified from Apify datacenter IPs and from a residential IP:

Engine	What it actually returns
Google	JS-required interstitial (soft block), on every IP tried
Yandex	HTTP 200, but redirected to showcaptchafast (SmartCaptcha)
Ecosia	HTTP 403 bot challenge

Getting past those needs a headless browser plus captcha solving — a different and much more expensive Actor. Rather than list three engines that bill you for a guaranteed-empty run, this Actor does not offer them at all. If you need Google, use Apify's Google Search Scraper — it is first-party, cheap and good, and this Actor is designed to sit alongside it, not to replace it.

Device targeting

The device input (desktop / mobile) is real, not a cosmetic toggle — but it only does something on the engine that actually responds differently to it:

Engine	device: mobile effect
Bing	Real, verified. Switches to a mobile browser User-Agent. This also helps get past Bing's JS bot-wall page that otherwise blocks datacenter IPs (mobile UA returned real results directly in testing; desktop UA needed the RSS fallback).
DuckDuckGo	No-op. The html.duckduckgo.com endpoint this Actor uses is a single, JS-free layout with no separate mobile version. The field is accepted but ignored.
Baidu / Yahoo	No-op. Not implemented. The field is accepted but ignored; it never causes an error.

Related searches & People Also Ask

Not implemented on any engine, and not a dataset field. On all four engines the related-searches and "People also ask" widgets are lazy-loaded by JavaScript after the initial response — the static HTML this Actor fetches never contains them, so there is nothing to parse. Adding the fields would only ever give you permanently-empty columns.

If you need People Also Ask, it is a Google feature in practice, and Apify's Google Search Scraper extracts it.

FAQ

Is it legal to scrape search engine results? This Actor reads only publicly available SERP data — what any visitor sees without logging in. No personal data behind authentication is touched. Review the target engines' terms and your local regulations for your specific use case.

Do I need an account or API key? No. Results are fetched from public SERP pages — no login, no API keys, no cookies.

Can I scrape Google with this? No — and that is deliberate, not a limitation being hidden. Google captcha-walls plain HTTP requests from every IP, so offering it would just bill you for empty runs. See Why Google, Yandex and Ecosia are not offered. Use Apify's Google Search Scraper for Google and this Actor for everything else.

Why does Yahoo return no results? Yahoo is the one engine here that needs a proxy: it returns nothing from Apify's datacenter IPs. Enable Apify Proxy with the RESIDENTIAL group in proxyConfiguration. Bing, DuckDuckGo and Baidu need no proxy at all.

How many results can I get? Up to maxPagesPerQuery pages per engine per query (~10 results/page, ~25 for DuckDuckGo), capped by maxItems (default 100, max 1000). The cap is shared out evenly across your queries, so every query gets results — it is never drained by whichever query happens to run first. Each result is billed as one dataset item — see Pricing above.

Are ads included? Only when includeAds is on; ad rows are tagged "ad": true and never consume organic ranks.

Can I get mobile search results? Yes for Bing: set device to mobile. It's a no-op on DuckDuckGo, Baidu and Yahoo — see Device targeting.

Does this Actor extract related searches or People Also Ask? No — on all four engines those widgets are JavaScript-rendered after page load, so a plain HTTP fetch never receives them. See Related searches & People Also Ask.

Something broken or missing? Search engines change their markup often. Open an issue on the Actor's Issues tab — it is monitored and reliability fixes ship fast.

Attribution

SERP URL construction and CSS selectors are a Python port of openserp by Rustem Kamalov, MIT License. See LICENSE and NOTICE.

Is this Actor useful to you? A quick ⭐ review on the Actor's Reviews tab helps other SERP and search-data users find it — and tells us what to build next.

Related Actors

• AI Job Search Agent — Web Job Finder (BYOK)
• LinkedIn Jobs Scraper — No Login, No Cookies
• Hacker News Who's Hiring Scraper