Bing Search Results Scraper (SERP)

Scrapes Bing organic search results — title, real destination URL, snippet, domain and absolute rank. No API key, no Azure subscription, no cognitive-services quota.

Useful for SEO rank tracking, competitor visibility monitoring, lead and prospect discovery, content research, and building link lists at scale.

What makes this different

Every URL is the real destination. Bing wraps every result link in a bing.com/ck/a?... redirect tracker. Scrapers that skip the decoding step return a dataset where every single row points at bing.com — technically 200 rows, practically zero. This Actor decodes the wrapper, so url is the site Bing actually found and domain is ready to group or filter on.

Pagination that actually paginates. Requesting page 2 on a cold session does not return page 2 — Bing serves page 1 again, with HTTP 200 and no error at all. A scraper missing the session warm-up looks like it works and quietly returns the same ten results over and over. This Actor warms the session first, then verifies each page contains genuinely new URLs and stops the moment it doesn't, so you are never charged for repeats.

Absolute rank, not per-page rank. rank counts across pages — result 1 of page 3 is rank 21. Non-organic cards are dropped before ranking, so ranks stay contiguous and are directly usable for SEO position tracking.

Video carousels don't pollute the dataset. Bing mixes video cards into the same result list as organic results, and they decode to internal Bing paths rather than external sites. They're excluded rather than emitted as rows pointing back at Bing.

Snippet and date are separated. Bing prefixes many snippets with a publication date (Aug 3, 2026 · In this section…). Left alone, that corrupts the snippet text. Here published_date carries the date and snippet is the clean description.

total_results_estimate on every row — Bing's own match estimate for the query, useful as a keyword-volume proxy without a second request.

Input

Field	Type	Notes
queries	array	One or more queries; each row records which produced it.
maxItems	integer	Results per query. Pages 10 at a time, stops early on repeats.
proxyConfiguration	object	Recommended. Also how you target a region — see below.

Region targeting, and why there's no "market" input

Bing accepts mkt / setlang / cc parameters, and they silently corrupt multi-word queries: with a market parameter set, best crm software returns results for best (Best Buy, a dictionary entry) and python web scraping tutorial returns results for python. Worse, the setting persists into the session cookie, so every later query on that session stays broken even after the parameter is removed.

An Actor exposing a "market" dropdown built on those parameters would return confidently wrong data for most real queries. So this Actor does not use them. Region targeting is done by proxy country instead — Bing localises results by IP, which works correctly — so pick the country in proxyConfiguration and you get that country's results with the query intact.

Output

{
  "rank": 1,
  "query": "best crm software",
  "page": 1,
  "title": "The Best CRM Software We've Tested for 2026 | PCMag",
  "url": "https://www.pcmag.com/picks/the-best-crm-software",
  "domain": "pcmag.com",
  "display_url": "https://www.pcmag.com › picks",
  "snippet": "We've tested the top CRM platforms to help you find the right one...",
  "published_date": "Aug 3, 2026",
  "total_results_estimate": 21400
}

Each run also writes a RUN_COVERAGE record to the key-value store with what was requested, what came back, and any per-query failures — so a partial run is visible rather than silent.

Notes on depth

Bing serves 10 organic results per page and ignores any larger count, so depth comes from paging. Real usable depth varies by query: broad commercial queries page freely, while narrow ones start repeating after a few pages. The Actor detects the repeat and stops there rather than billing you for duplicates.

If Bing serves a challenge page during a long run, that query fails with a clear message rather than returning partial garbage. Enabling a proxy and running fewer queries per run resolves it.

Local development

pip install -r requirements.txt
python test_local.py "best crm software" --max 25 --out sample_output.json
python test_local.py "python web scraping tutorial" "best crm software" --max 25

sample_output.json is real output from a live two-query run.