Bing Search Scraper: News, Publishers & Real Links

Bing Search Scraper searches Bing News by keyword across 60 country/language markets and returns one clean row per story: headline, publisher name, snippet, publish date in ISO 8601, a thumbnail where one exists, and the publisher's own URL rather than the click-tracking redirect — decoded on the spot, with no extra request and no extra charge. Optionally it also returns Bing's top-10 web results for the same keywords.

No API key, no quota, no proxy setup and no browser.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/bing-search-scraper.md

What you get

Output field	Meaning	Null when
title	The headline, or the web page's title	never
url	The publisher's own address. On news rows it is decoded out of the redirect; on web rows it is the link as given	never
redirectUrl	The source's click-tracking link, kept so you can audit the decode	always on web rows — that surface has no redirect
domain	Host of url, lowercased and without www. — the field you group by	never
snippet	Real prose from the source. Not a restatement of the title	never
publisher	The outlet the story belongs to, including when the link resolves to an aggregator	always on web rows — that surface names no source
publishedAt	ISO 8601, UTC	always on web rows — see Limits
publishedRaw	The source's own date string, exactly as sent. This is where a web row's date lives, unpromoted	never
thumbnailUrl	Story image, delivered in the same response	on web rows, and on roughly two news rows in five
thumbnailWidth	Image width in pixels	wherever thumbnailUrl is null
thumbnailHeight	Image height in pixels	wherever thumbnailUrl is null
surface	news or web	never
query	Which keyword produced this row	never
market	The market actually used — en-US on every web row regardless of input	never
position	Rank within its keyword and surface	never
scrapedAt	Run timestamp	never

A RUN_SUMMARY record in the key-value store holds the market used, the surfaces run, every keyword requested and every one rejected with the reason, requests made, results saved per surface, pages fetched per keyword, duplicates collapsed, the date range applied and how many rows it removed, how many web rows the relevance check dropped and how many keywords the web surface could not answer at all, the 10-result web ceiling stated outright, and any field that failed its shape check. Failures are named there rather than left to look like a quiet news day.

Input

Field	Type	Default	Notes
queries	array	required	Keywords, one per line. Plain terms only — a "quoted phrase" works, but site:, intitle:, inurl:, -exclude and OR are not supported by this source and are rejected by name before the run
surfaces	array	["news"]	news, web, or both. News is the default because it is the surface that answers reliably
market	string	en-US	Country and language together, from one value. 60 available, each verified to answer in its own language. Applies to news; the web surface ignores it and is always read as en-US
maxResultsPerQuery	integer	20	Cap per keyword per surface, and the main cost control. News pages to roughly 100 results per keyword. The web surface returns at most 10 per keyword no matter what this is set to — that surface has no page 2
freshness	string	any	any, day or week. Narrows the news request itself; the boundaries are the source's own and approximate. For an exact cutoff use dateFrom
dateFrom	string		2026-01-15, or relative: 7 days, 36 hours, 3 months. Applied to publishedAt after the fetch. News rows only
dateTo	string		Same formats. Ignored unless dateFrom is also set, and the log says so
deduplicateBy	string	url	url, title or none. Collapses across the whole run and per surface, so a result two keywords both return is saved and charged once. Use none for one row per keyword regardless

queries is required; a run with an empty list fails with a named error rather than returning nothing.

{
  "queries": ["semiconductor export controls", "bitcoin etf"],
  "surfaces": ["news"],
  "market": "de-DE",
  "maxResultsPerQuery": 40,
  "freshness": "week"
}

Output

One record per result. Every field is present on every record, and null always means the value is not available for that row — never that the column was quietly dropped. Where a field is null on a whole surface, the table above says so and Limits explains why.

{
  "title": "Artificial intelligence used to design brand new viruses",
  "url": "https://www.msn.com/en-us/news/technology/artificial-intelligence-used-to-design-brand-new-viruses/ar-AA29xZlR",
  "redirectUrl": "http://www.bing.com/news/apiclick.aspx?ref=FexRss&url=https%3a%2f%2fwww.msn.com%2fen-us%2fnews%2ftechnology%2fartificial-intelligence-used-to-design-brand-new-viruses%2far-AA29xZlR&…",
  "domain": "msn.com",
  "snippet": "Scientists made 16 successful viruses that had their genetic code designed by artificial intelligence.",
  "publisher": "BBC on MSN",
  "publishedAt": "2026-08-06T11:01:52.000Z",
  "publishedRaw": "Thu, 06 Aug 2026 11:01:52 GMT",
  "thumbnailUrl": "http://www.bing.com/th?id=ONUT.wO-W4yL5hEVfZv3ss1TPJQ&pid=News",
  "thumbnailWidth": 2641,
  "thumbnailHeight": 1485,
  "surface": "news",
  "query": "artificial intelligence",
  "market": "en-US",
  "position": 1,
  "scrapedAt": "2026-08-09T12:04:11.000Z"
}

Use cases

• Multi-market brand monitoring. Track a brand across 60 national news markets on one schedule, each answering in its own language from its own outlets, and diff successive runs on url.
• Keyword enrichment for agents. Ten to forty current, dated, sourced headlines for each of thousands of keywords, in one call, with a real publisher link — the shape a retrieval step actually wants.
• A thumbnailed news corpus without a second fetch. The image arrives in the same response as the headline, so building an illustrated corpus costs one request per page of results rather than one per story.
• A second index alongside Google News. Different ranking, different publisher mix, same row shape — run both and compare domain distributions for the same keyword.
• Top-10 web results for a keyword list. With surfaces: ["web"], ten organic results per keyword and nothing you have to configure. Read the Limits section first.

Limits and behaviour

• The web surface returns at most 10 results per keyword, and there is no page 2. Every paging parameter this source accepts was verified to return the identical ten rows — 0 new results on every attempt. Raising maxResultsPerQuery does not change it and the Actor never issues a second web request for one keyword. This Actor is not a rank tracker and cannot be used as one.
• The web surface is en-US only, and answers roughly four keywords in ten. Every other market returned unrelated results in testing, so the surface is pinned and every web row is stamped market: "en-US" whatever you asked for. Worse, when it cannot answer a keyword it does not say so — it returns ten well-formed rows about something else entirely, with the response echoing your keyword back.
• When the web surface cannot answer a multi-word keyword, it collapses it to the first word. This is the failure mode that matters, because it does not look like one. Asked for best running shoes for flat feet it returned ten dictionary definitions of the word best; for electric vehicle tax credit, a rural electric cooperative in Kansas; for commercial real estate loans, a bank in Qatar; for apache kafka consumer group, Apache HTTP Server downloads. Every one of those rows contains a word from your keyword, so a check for "mentions any of your terms" would pass all ten and charge you for all ten. This Actor therefore requires a web row to mention at least two distinct terms from your keyword before it is saved — or the only term, if your keyword is a single word. Measured over 20 keywords, that rule cut delivered rows from 160 to 48 while keeping every row for the keywords the surface genuinely answered. Dropped rows are dropped before they reach the dataset, so they are never charged, and RUN_SUMMARY reports how many were dropped and how many keywords produced nothing. A keyword this surface cannot answer costs you nothing at all.
• Some broad news keywords return nothing. A handful of very general topic phrases — news, business news, world news, sports news, local news, tech news, finance news, politics news — are answered with a topic landing page rather than a result set, so they produce zero news rows and are named as a failure in RUN_SUMMARY.failures rather than silently returning nothing. Nothing is charged for them. Use a specific subject instead (semiconductor exports, bitcoin etf, a company or person's name), which is what this surface is good at.
• Search operators are not supported and are rejected up front. site:, intitle:, inurl:, -exclude, OR and parenthesised groups return unrelated results rather than an error. A keyword containing one is named in RUN_SUMMARY.rejectedInputs and skipped; the rest of the run proceeds. A "quoted phrase" does work.
• News reaches roughly 100 results per keyword before repeating. The run pages until a page adds nothing new, then stops and reports how many pages it used. A cap above that returns what exists rather than padding.
• thumbnailUrl is null on roughly two news rows in five, because the source carries no image for those stories — measured at 58% present across 1,400 items. It is null on every web row, because that surface carries no image element at all. Width and height are present exactly where the image is.
• Roughly one news link in seven resolves to msn.com rather than the originating outlet, which makes MSN the single most common host by a wide margin. That is syndication, and the information is not lost: publisher names the real outlet ("BBC on MSN", "Yahoo Finance on MSN") while domain shows where the page is actually hosted. If you group by domain alone, MSN will look like your top publisher — group by publisher instead.
• A web row has no publisher and no verified publish date. The web surface carries no source element, so publisher is null. It does send a date, and that date is not a publish date: on a single page of ten results it spanned three years with no relation to the pages' own content. It is the source's internal stamp, so it stays in publishedRaw, labelled, and publishedAt is null. A populated column that is wrong is worse than an honest empty one.
• freshness boundaries are the source's own and approximate. day measured a little over 24 hours and week a little over six days on one keyword, and they shift between keywords. For an exact cutoff use dateFrom, which is applied to publishedAt after the fetch and therefore means one thing regardless.
• A market cannot be invented. An unrecognised market is not rejected by the source — it quietly answers with some other country's results. Only markets verified to answer in their own language are offered, and anything else fails before a single request.
• One bad input never ends the run. A rejected keyword lands in RUN_SUMMARY.rejectedInputs and the rest proceeds. A keyword that simply matched nothing is not a failure either: the run finishes with an empty dataset and says why.
• Headlines and metadata only. There is no article full text here and no way to search inside one.
• Public data only. No credentials, and nothing that works around access controls.

FAQ

Why did a keyword return zero web results? Because fewer than two of your terms appeared in any of the ten rows the surface returned. That surface answers roughly four en-US keywords in ten; for the rest it collapses your keyword to its first word and returns a full page about that instead — ten definitions of best, say, for best running shoes for flat feet. Those rows all contain a word from your keyword, which is exactly why the check requires two. You were not charged for the ones that were dropped, and RUN_SUMMARY.webKeywordsUnanswered counts the keywords it happened to. The news surface behaves differently: it returns nothing at all when it has nothing.

Why did a keyword return zero news results? Usually the keyword genuinely has no current coverage — that is the source being honest, and the run still exits successfully. The exception is a very broad topic phrase such as news, business news or world news, which is answered with a topic landing page rather than a result set; those are named in RUN_SUMMARY.failures so you can tell the two apart, and neither costs you anything.

Why is publishedAt null on my web rows? Because the web surface's date is not a publish date. It spanned three years across one page of ten results in testing, with no relation to the pages themselves. Rather than fill a column with something wrong, it stays in publishedRaw where it is clearly labelled as the source's own string.

Why are so many links msn.com? Syndication. Roughly one news story in seven is carried on MSN rather than the originating outlet's own site, which is enough to make it the top host in most runs. publisher still names the real outlet, so group by publisher rather than domain when you want the newsroom rather than the host.

Can I use this to track my Bing rankings? No, and it is worth being precise about why: this surface has exactly one page. Ten results, no page 2, verified against every paging parameter it accepts. Anything sold as a Bing rank tracker on this data is page-1-only data under a page-N name. Use the news surface for coverage monitoring, which is what this Actor is built for.

Do search operators work? No. This source answers site:apify.com with unrelated pages rather than an error, so any query containing an operator is rejected by name before the run instead of quietly returning nonsense. Quoted phrases do work.

Why do I get fewer results than I asked for? Four possibilities, and RUN_SUMMARY.perQuery tells you which: the keyword has less coverage than your cap; you hit a surface ceiling (web stops at 10 always, news stops when a page adds nothing new); dateFrom/dateTo removed rows after the fetch, counted in dateRange.newsRowsOutOfRange; or deduplicateBy collapsed a result two of your keywords both returned, counted in duplicatesCollapsed. You are charged only for the rows you actually receive in every one of those cases.

How do I run this on a schedule? Apify schedules, or your own cron against the API. Keep the dataset and diff runs on url, which is stable per result.

Can I plug it into something else? Yes. Apify API, the client libraries, webhooks, scheduled runs, dataset exports to JSON, CSV or Excel, or MCP. The output is structured JSON.

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~bing-search-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "queries": ["semiconductor export controls"],
 "surfaces": ["news"],
 "market": "de-DE",
 "maxResultsPerQuery": 40,
 "freshness": "week"
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/bing-search-scraper').call({
 queries: ['bitcoin etf', 'semiconductor export controls'],
 surfaces: ['news'],
 market: 'en-US',
 maxResultsPerQuery: 40,
 dateFrom: '7 days',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const r of items) {
 console.log(`${r.publishedAt} ${r.publisher} — ${r.title}`);
 console.log(r.url);
}