# Bing Search Scraper: Organic Results, Ads & Related Searches (`rjmacarthy/bing-search-scraper`) Actor Scrape Bing results pages: organic results with decoded real URLs, sitelinks, text and shopping ads, Copilot AI answer, People also ask, related searches and result counts. Markets, mobile, date filters. Pay per results page, not per result. - **URL**: https://apify.com/rjmacarthy/bing-search-scraper.md - **Developed by:** [rj](https://apify.com/rjmacarthy) (community) - **Categories:** SEO tools, Automation - **Security:** Limited permissions - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 results page scrapeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Bing Search Scraper: Organic Results, Ads & Related Searches **Bing Search Scraper** turns Bing results pages into clean JSON: **organic results with the real destination URLs** (Bing's `bing.com/ck/a?...` redirect links are decoded), **sitelinks**, **text ads and shopping ads** with landing pages, Bing's **Copilot AI answer** with its sources, **People also ask**, **related searches**, answer boxes and the total results estimate. Choose the market, device, date range and SafeSearch, and scrape up to 10 pages per query. Most Bing scrapers charge for every organic result and return only titles, links and snippets. This one charges **per results page** ($0.002 for up to 10 organic results plus every SERP feature on the page), so SEO rank tracking costs a fraction of per-result pricing. It also checks every page: Bing quietly serves bots a degraded page with results for the wrong query, and the Actor detects that, retries in a fresh session and **never charges for it**. ### What data can you get from Bing? | Section | Fields | |---|---| | **Organic results** | `position` (overall rank across pages), `rankOnPage`, `title`, `url` (real URL, decoded), `displayUrl`, `domain`, `siteName`, `snippet`, `date` (ISO) + `dateText`, `sitelinks[{title, url, snippet}]` | | **Text ads** | `placement` (top / middle / bottom / sidebar), `position`, `title`, `url` (landing page, decoded), `displayUrl`, `domain`, `advertiser`, `description`, `callouts`, `sitelinks[{title, url}]` | | **Shopping ads** | `title`, `price`, `merchant`, `shipping`, `url`, `domain`, `placement` | | **Copilot AI answer** | `aiAnswer.text` (the generated summary) and `aiAnswer.sources[{title, url}]` | | **Answer box / knowledge panel** | `answerBox.type` (fact / entity / answer), `title`, `subtitle`, `answer`, `url` | | **People also ask** | `question`, `sourceUrl`, `answer` (usually `null`, because Bing loads answers on click) | | **Related searches** | `query`, `url` | | **Page info** | `query`, `page`, `url` of the results page, `totalResults` (number) + `totalResultsText`, `marketDetected` / `regionDetected` (what Bing actually used), `spellingCorrection`, counts of every section, `duplicatesRemoved`, `warnings`, `scrapedAt` | Every page item has the same keys (`null` or `[]` when a section isn't on the page). ### Use cases - **SEO rank tracking on Bing.** Track where your pages rank for 100s of keywords, by market and device, on a schedule. Positions are de-duplicated and numbered across pages (1-100). - **Competitor and ad intelligence.** See who bids on your keywords, their ad copy, sitelinks, callouts and landing pages, and which products show in shopping ads. - **AI visibility (GEO).** See what Bing's Copilot answer says for your topics and which sources it cites. - **Content research.** Mine People also ask and related searches for content ideas and FAQs. - **Market research and lead lists of websites** (domains, not personal data) for any topic and country. - **Feeding LLM agents and RAG pipelines** with fresh web search results, via the API or MCP. ### How to use it 1. Click **Try for free**. 2. Enter one or more **Search queries** (one per line). 3. Choose **Results pages per query** (up to 10; page 1 holds up to 10 organic results, often fewer on commercial queries, so use 2-3 pages for a top-10/top-20 ranking), the **Market** and, optionally, device, date range, SafeSearch or a site restriction. 4. Click **Start** and download the results as JSON, CSV, Excel or HTML, or read them through the API. #### Input examples Track the first three results pages for three keywords in the UK: ```json { "queries": ["project management software", "best crm for small business", "kanban board app"], "maxPagesPerQuery": 3, "market": "en-GB", "device": "desktop" } ``` German results from the past week, one row per organic result for spreadsheets: ```json { "queries": ["künstliche intelligenz gesetz"], "maxPagesPerQuery": 2, "market": "de-DE", "freshness": "week", "outputFormat": "flat" } ``` Mobile results through Apify Proxy (US IPs): ```json { "queries": ["pizza delivery", "running shoes"], "device": "mobile", "market": "en-US", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyCountry": "US" } } ``` | Input | Default | Notes | |---|---|---| | `queries` | `project management software`, `eiffel tower`, `best running shoes` | Required. Exactly what you'd type into Bing; quoted phrases work. Up to 1,000 per run. | | `maxPagesPerQuery` | 1 | 1–10. Stops early (without charging) when Bing has no more new results. | | `market` | `en-US` | 41 markets (en-GB, de-DE, fr-FR, es-MX, ja-JP, ...). See *Tips* for how en-US works. | | `language` | market's language | Bing interface language (`setlang`), e.g. `en` or `pt-BR`. | | `device` | `desktop` | `mobile` scrapes the mobile results page (page 1 only). | | `safeSearch` | `moderate` | `strict` or `off`. | | `freshness` | `any` | `day`, `week` or `month`. | | `site` | – | Adds `site:example.com` to every query (see *Tips*). | | `outputFormat` | `flat` | `flat`: one item per results page plus one row per organic result (rows are free). `page`: page items only. | | `maxConcurrency` | 2 | Queries scraped in parallel (1–3). | | `proxyConfiguration` | no proxy | Apify Proxy (datacenter) recommended for more than ~100 pages per run. Residential is not supported. | ### Output example One results page (real output, shortened): ```json { "id": "bing:page:1080cccf0aebe24b", "type": "page", "query": "project management software", "page": 1, "url": "https://www.bing.com/search?q=project+management+software&form=QBLH", "device": "desktop", "market": "en-US", "marketDetected": "en-US", "totalResults": 26800, "totalResultsText": "26,800 results", "organicCount": 3, "adsCount": 6, "organicResults": [ { "position": 1, "rankOnPage": 1, "title": "10 Best Project Management Software Of 2026", "url": "https://www.forbes.com/advisor/business/software/best-project-management-software/", "displayUrl": "https://www.forbes.com › ... › software › best-project-management-software", "domain": "forbes.com", "siteName": "Forbes", "snippet": "Compare 22 providers of project management software based on pricing, features, integrations and customer …", "date": "2026-09-25", "dateText": "Sep 25, 2026", "sitelinks": [ { "title": "Airtable Review", "url": "https://www.forbes.com/advisor/business/software/airtable-review/", "snippet": "Airtable is a cloud-based project management software that allows teams to store, organize …" } ] } ], "ads": [ { "position": 1, "placement": "top", "title": "Asana for Project Management - asana.com - Get Started for Free", "url": "https://asana.com/uses/project-management?utm_campaign=...", "displayUrl": "https://asana.com › Asana › Manage-projects", "domain": "asana.com", "advertiser": "Asana", "description": "Millions of users in 190 countries trust Asana to help them manage work. ...", "callouts": null, "sitelinks": [{ "title": "Pricing & Plans", "url": "https://asana.com/pricing?utm_campaign=..." }] } ], "aiAnswer": { "text": "Project management software helps teams plan, track, and deliver projects efficiently, with top tools in 2026 including ClickUp, monday.com, Jira, and ProofHub. ...", "sources": [{ "title": "10 Best Project Management Software Of 2026", "url": "https://www.forbes.com/advisor/business/software/best-project-management-software/" }] }, "answerBox": null, "peopleAlsoAsk": [], "relatedSearches": [ { "query": "project management software comparison 2026", "url": "https://www.bing.com/search?q=project+management+software+comparison+2026&FORM=QSRE9" } ], "duplicatesRemoved": 0, "warnings": [], "scrapedAt": "2026-11-02T13:45:42.117Z" } ``` The full sample (four pages and a flat row) is in `samples/sample-output.json`. In the Apify Console the dataset has ready-made tables: **Results pages**, **Organic results**, **Ads**, **SERP features** and **Flat organic rows**. ### How much does it cost? You pay per **results page** that is successfully scraped. There's no start fee and no subscription. | Event | Price | When | |---|---|---| | `results-page` | **$0.002** | One Bing results page saved to the dataset (up to ~10 organic results plus all ads, AI answer, People also ask and related searches on that page). | Never charged: degraded or challenge pages from Bing, pages with no results, pages where Bing only repeats earlier results, and flat per-result rows (`outputFormat: "flat"`). Worked examples: - **1,000 results pages ≈ $2.00** (up to ~10,000 organic results). - **Daily rank tracking of 100 keywords, 3 pages each: 300 pages = $0.60 per day.** - **One results page for one keyword = $0.002**: up to 10 organic results (often fewer on commercial queries) with ads and SERP features included. Set **Maximum cost per run** on the run to cap your spend; the Actor stops cleanly when the budget is reached. ### Tips and limits (please read) - **Bing degrades automated traffic.** Bing rarely blocks with an error. Instead it returns a normal-looking page whose results only match the first word of your query, or belong to someone else's search. The Actor imitates a person searching from the bing.com home page, which avoids this in our tests, and it checks every page (layout signals and whether results mention your query). Degraded pages are retried in a new session (up to 3 tries) and never charged. If many pages are degraded, enable **Apify Proxy**. - **Use Apify Proxy for more than ~100 pages per run.** Each query then gets a fresh datacenter IP. Residential proxies are not supported (see the FAQ). - **Market and country.** For every market except en-US the Actor applies the market explicitly (Bing's `setmkt`). For **en-US**, forcing the market made Bing return degraded pages in our tests, so the Actor lets Bing choose the market from the IP address and language. On US IPs that is the US market; from elsewhere you get English results localised to the IP's country. For true US results, use Apify Proxy with country US. `marketDetected` and `regionDetected` show what Bing actually used. - **site: searches.** In our tests from datacenter IPs, Bing ignored `site:` restrictions for automated clients. The Actor detects this (results not on your domain) and doesn't charge, but the run may return nothing. If that happens, drop the restriction and filter by `domain` yourself. - **Mobile** scrapes page 1 only, because Bing's mobile page loads more results by infinite scroll. - **Results per page vary.** Bing decides how many organic results each page shows: up to 10, but commercial queries (and questions that get a Copilot answer) often get only 2–3 on page 1, because ads, shopping ads, the AI answer and other answer blocks fill the page. That is the same page a person sees, not a blocked one. Such pages carry a note in `warnings`, and `RUN_SUMMARY` counts them (`pagesWithFewOrganicResults`). - **No results are skipped between pages.** Pagination follows Bing's own "next page" offset, so if page 1 shows 2 results, page 2 starts at result 3. Positions continue across pages, and repeats are removed (`duplicatesRemoved`). - **Result count.** `totalResults` / `totalResultsText` appear only when Bing shows its "About N results" bar; Bing often hides it, and then they are `null` and the page's `warnings` says so. - **People also ask answers** are loaded by Bing only when a question is clicked, so `answer` is usually `null`. The question and its source URL are included. - **Speed.** Requests are spaced like a person browsing (1–3.5 s between pages, at most one request per second overall): roughly 1,000–2,000 pages per hour with the default 2 parallel queries. ### FAQ **Is it legal to scrape Bing?** This Actor only reads public search results pages, without logging in. It doesn't collect personal data such as e-mail addresses or phone numbers. You're responsible for how you use the data and for complying with applicable laws (e.g. copyright and GDPR) and Bing's terms. If in doubt, ask a lawyer. **Why did a query come back as "blocked"?** Bing served degraded or challenge pages three times in a row for that query. Nothing was charged. Enable Apify Proxy and run it again. `RUN_SUMMARY` in the key-value store lists the reason per query. **Do I need a proxy? Can I use residential proxies?** Usually no proxy is needed: the Actor works from Apify's own servers. For runs of more than ~100 pages, enable **Proxy configuration** with Apify Proxy's automatic (datacenter) proxies, so each query gets a fresh IP. Residential proxies are not supported: if you select the RESIDENTIAL group, the Actor logs a warning and uses Apify datacenter proxy instead. Your own proxy URLs (custom proxies) are used as given. **Why are there fewer than 10 organic results on a page?** That's what Bing showed: on commercial queries, ads, shopping carousels and the AI answer push organic results down, and Bing's own "next page" link continues right after them. The page's `warnings` field says so. Scrape 2–3 pages per query for a full top-10 or top-20 ranking. **Can I get the real URL instead of `bing.com/ck/a?...`?** Yes, always. Organic links, sitelinks, related searches and ad landing pages are decoded. **Does it support Bing News, Images or Videos?** No, only web results pages. ### Integrations - **API:** start runs and read the dataset with the Apify API or the JavaScript / Python clients (Actor ID `rjmacarthy/bing-search-scraper`). - **Make, Zapier, n8n:** use the Apify integrations to run the Actor and push results into Google Sheets, Airtable, Slack or your CRM. - **Schedules and webhooks:** run daily for rank tracking and get notified when a run finishes. - **MCP / AI agents:** add the Actor to Claude, Cursor or another MCP client through the Apify MCP server, and let agents search Bing. ### Changelog - **0.1 update (November 2026):** Residential proxy groups fall back to Apify datacenter proxy with a warning. Default output is now `flat` (page items plus free per-result rows). Pagination follows Bing's own next-page offset (on pages where Bing shows fewer than 10 organic results, page 2 no longer skips results). Pages with fewer than 5 organic results or no result count are flagged in `warnings`, the run summary and the status message. - **0.1 (November 2026):** First release. Organic results with decoded URLs and sitelinks, text and shopping ads, Copilot AI answer, People also ask, related searches, answer box, 41 markets, mobile, date range, SafeSearch, site restriction, flat output, degraded-page detection with retries, migration-safe progress. ### More scrapers by the same developer - [Google Ads Transparency Scraper](https://apify.com/rjmacarthy/google-ads-transparency-scraper) - [AI Visibility Tracker: ChatGPT, Perplexity, Gemini, Claude](https://apify.com/rjmacarthy/ai-search-visibility-tracker) - [Substack Scraper](https://apify.com/rjmacarthy/substack-scraper) - [Telegram Channel Scraper](https://apify.com/rjmacarthy/telegram-channel-scraper) - [ATS Jobs Scraper: Greenhouse, Lever, Ashby & Workday](https://apify.com/rjmacarthy/ats-jobs-scraper) - [Dice Jobs Scraper](https://apify.com/rjmacarthy/dice-jobs-scraper) Browse all my Actors on [my Apify Store profile](https://apify.com/rjmacarthy). # Actor input Schema ## `queries` (type: `array`): One query per line, exactly as you would type it into Bing (operators like "quoted phrases" work). Each query is scraped in its own Bing session. ## `maxPagesPerQuery` (type: `integer`): How many results pages to scrape for each query. Bing decides how many organic results a page shows: up to 10, but often only 2-6 on page 1 of commercial queries, where ads, shopping and answers take the space. Each next page continues exactly where Bing's own pager does, so no result is skipped. Each scraped page is one billable "results-page" event. The Actor stops early, without charging, when Bing runs out of new results. ## `market` (type: `string`): Bing market to search in. Sets the country and the language Bing uses to rank results. For United States (en-US) Bing picks the market from the IP address and language, so for US results run from US IPs (Apify Proxy, country US); other markets are applied explicitly. ## `language` (type: `string`): Bing interface language (setlang), e.g. "en", "de" or "pt-BR". Leave empty to use the market's language. ## `device` (type: `string`): Scrape the desktop or the mobile results page (mobile uses an Android Chrome user agent). Bing's mobile page has no classic pagination, so mobile scrapes page 1 only. ## `safeSearch` (type: `string`): Bing SafeSearch level for adult content. ## `freshness` (type: `string`): Only results Bing considers published or updated in this period. ## `site` (type: `string`): Only results from this domain, e.g. "example.com" or "example.com/blog" (adds site: to every query). Note: in our tests Bing degraded site: searches for automated clients from datacenter IPs; such pages are detected, retried and never charged. If that happens, leave this empty and filter the results by domain yourself. ## `outputFormat` (type: `string`): "One item per results page" puts organic results, ads, People also ask etc. in nested arrays (one dataset item per page). "Page items + one row per organic result" additionally outputs every organic result as its own flat row (type "organic"), handy for spreadsheets; those rows are free - you still pay per page. ## `maxConcurrency` (type: `integer`): How many queries run at the same time (each in its own Bing session). Requests are spaced at least 1 second apart in total. ## `debugSaveHtml` (type: `boolean`): Diagnostics only: store every raw Bing response in the run's key-value store (DEBUG-\* records). ## `proxyConfiguration` (type: `object`): Optional. Recommended for runs of more than ~100 pages: with Apify Proxy (datacenter) each query gets a fresh IP. Degraded or challenged pages are retried with a new session and never charged. Residential proxies are not supported: if you select the RESIDENTIAL group, the Actor logs a warning and uses Apify datacenter proxy instead. Your own proxy URLs are used as given. ## Actor input object example ```json { "queries": [ "project management software", "eiffel tower", "best running shoes" ], "maxPagesPerQuery": 1, "market": "en-US", "device": "desktop", "safeSearch": "moderate", "freshness": "any", "outputFormat": "flat", "maxConcurrency": 2, "debugSaveHtml": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `pages` (type: `string`): One row per results page: query, page, total results and counts of each SERP feature. ## `organic` (type: `string`): Every organic result (position, title, decoded URL, snippet, date, sitelinks). ## `ads` (type: `string`): Text ads with placement, advertiser and landing page. ## `allFields` (type: `string`): Complete page items with all nested arrays. ## `summary` (type: `string`): Per-query status, pages, blocked pages, request statistics. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "project management software", "eiffel tower", "best running shoes" ], "maxPagesPerQuery": 1, "market": "en-US", "outputFormat": "flat", "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("rjmacarthy/bing-search-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": [ "project management software", "eiffel tower", "best running shoes", ], "maxPagesPerQuery": 1, "market": "en-US", "outputFormat": "flat", "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("rjmacarthy/bing-search-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "project management software", "eiffel tower", "best running shoes" ], "maxPagesPerQuery": 1, "market": "en-US", "outputFormat": "flat", "proxyConfiguration": { "useApifyProxy": false } }' | apify call rjmacarthy/bing-search-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,rjmacarthy/bing-search-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/OwWS7qxM4DPV0K25N/builds/VXyHPY3fqn8NK8F54/openapi.json