Google News to clean article text for AI/RAG

This Apify actor get back all plain text news for your Agentic AI. Keep update your AI Agent with latest news! It searches Google News RSS for one or more country/language editions, collects article URLs, opens each news page, extracts clean text, and stores one dataset item per processed URL. It uses Google News RSS discovery plus Playwright-based extraction with automatic HTTP fallback when browser rendering fails.

What it does

• Accepts a Google News search query, including advanced operators such as intitle, inurl, site, -, quoted exact matches, AND, and OR.
• Accepts a predefined Google News topic as an alternative to searchQuery. Exactly one between searchQuery and topic must be provided.
• Accepts one or more languageAndRegion pairs in the format CC:ll, for example IT:it, US:en, LT:lt, and uses them to build Google News RSS parameters gl, hl, and ceid.
• Searches Google News RSS for each selected edition and decodes Google News article links into publisher URLs using googlenewsdecoder.
• Limits discovery to a maximum number of unique news sites per language/region pair through maxSites.
• Uses daysBack as the RSS search window and sets chunkDays = daysBack in the final actor logic.
• Opens discovered article URLs with Playwright, extracts main page text, and falls back to HTTP + BeautifulSoup text extraction if Playwright fails.
• Builds an Accept-Language header from the selected edition language and includes English as fallback for non-English editions.
• Processes multiple pages in parallel using maxConcurrency, capped to 50 open Playwright pages at a time.
• Writes one dataset item per processed URL and emits detailed performance logs for RSS discovery, URL decoding, browser extraction, fallback extraction, and overall actor runtime.

Input

Advanced search filters

You can also use advanced search operators in your queries, such as intitle, inurl, site, exclude operator -, exact match with double-quotes "", AND, OR and more.

Example queries with advanced operators

Query	Explained
intitle:"AI" AND site:bbc.com	Finds articles with "AI" in the title from BBC.
site:reuters.com "stock market" -crypto	Finds stock market articles on Reuters, excluding crypto-related ones.
"Samsung Galaxy S25" AND (review OR comparison)	Searches for reviews or comparisons of Samsung Galaxy S25.
site:nytimes.com intitle:"election" after:2025-01-01	Retrieves recent NY Times articles with "election" keyword in the title.
inurl:blog OR inurl:news "climate change"	Searches for climate change mentions in blog or news URLs.

For more information, see the Google Guide on Search Operators.

Example queries with advanced operators

You can use predefined Topic. Be sure to use Topic or Search query. Do not use Together. See example table with all behavior.

topic	searchQuery	Comportamento finale
DO_NOT_USE_TOPIC	empty	Exit without error.
DO_NOT_USE_TOPIC	real query	it will use searchQuery.
TECHNOLOGY	empty	it will use topic.
TECHNOLOGY	real query	Errore: You must specify exactly one of 'searchQuery' and 'topic'.
WORLD	empty	Usa topic.
BUSINESS	empty	Usa topic.
HEALTH	empty	Usa topic.

Input fields

Field	Type	Required	Description
searchQuery	string	Conditionally	Query written as for the Google News search bar. Advanced operators are supported. Use searchQuery as an alternative to topic. Exactly one between searchQuery and topic must be provided.
topic	string	Conditionally	Predefined Google News topic used as an alternative to searchQuery. Exactly one between searchQuery and topic must be provided. Supported topic values are listed below.
maxConcurrency	integer	No	Maximum number of pages opened simultaneously in the Playwright browser. Capped to 50. This value represents the maximum number of browser pages or tabs processed in parallel, not the number of system threads. Max value is capped to 50.
languageAndRegion	array of strings	Yes	One or more country/language pairs in the format CC:ll, for example IT:it, US:en, LT:lt. If you specify more then one pair foreach pair will be returned maxSites site back. Practical whitelist of commonly used Google News country/language pairs in the format CC:ll. This is not an official or complete list of all combinations supported by Google News.
maxSites	integer	No	Maximum number of unique news sites returned for each language/region pair. RSS feed finds more results, but domain deduplication filter stops when it reaches maxSites unique domains.
daysBack	integer	No	Number of days back to search in Google News RSS. How many days back to search. chunkDays is always set equal to daysBack.
decodeInterval	integer	No	Interval passed to googlenewsdecoder. Default is 1. This is used to time the decoding of Google News links: it controls how often the decoder attempts to resolve the URL before considering it failed or moving on to the next attempt.

In your case, a value of 1 essentially indicates a very short interval between attempts, useful for not overloading the service and reducing the risk of errors or rate limits. Increasing the value slows the process but can be more stable when decoding many links in sequence. |

Example input

Search query example:

{
    "daysBack": 100,
    "decodeInterval": 1,
    "languageAndRegion": [
        "US:en"
    ],
    "maxConcurrency": 50,
    "maxSites": 10,
    "searchQuery": "jannik sinner"
}

Topic example:

{
    "daysBack": 100,
    "decodeInterval": 1,
    "languageAndRegion": [
        "US:en"
    ],
    "maxConcurrency": 50,
    "maxSites": 10,
    "topic": "TECHNOLOGY"
}

Supported topics

Topic	Meaning
WORLD	International news.
NATION	National news for the selected country/region edition.
BUSINESS	Business, markets, companies, and economy coverage.
TECHNOLOGY	Technology, digital products, software, internet, and innovation coverage.
ENTERTAINMENT	Entertainment, media, celebrities, film, music, and culture coverage.
SPORTS	Sports news and related coverage.
SCIENCE	Science and research coverage.
HEALTH	Health, medicine, and public health coverage.

How it works

The actor runs in two stages. First, for each selected language/region pair it either queries Google News RSS search using searchQuery and the after: and before: filters derived from daysBack, or it loads the corresponding Google News topic feed when topic is provided. It then decodes Google News links into publisher URLs and deduplicates the results.

Second, it opens each discovered news URL with Playwright, attempts to extract the main text from article-like selectors such as article, main, and common content containers, and falls back to a plain HTTP request plus HTML text cleanup when browser extraction fails.

Concurrency

The actor uses a shared Playwright browser and limits concurrent page work with asyncio.Semaphore(maxConcurrency), so maxConcurrency represents the maximum number of browser pages or tabs processed in parallel, not the number of OS threads.

This makes concurrency easy to reason about operationally: if maxConcurrency is set to 10, the actor will process up to ten pages at the same time in Chromium.

Language handling

For each languageAndRegion pair, the actor extracts the language part and builds the Accept-Language header used for both Playwright and HTTP fallback requests. For non-English editions it uses a language list such as ["it", "en"], which becomes a header like it,en;q=0.9; for English editions it uses ["en"].

This improves localization hints to publisher sites, but the final language of the returned page still depends on how each site handles geographic and language negotiation.

Output

Each dataset item contains the processing result for one discovered publisher URL.

Successful fallback result

{
    "position": 1,
    "title": "Breaking: T-Mobile To Force Migrate Over 8 Million Customers To More Expensive Plans - The Mobile Report",
    "domain": "http://tmo.report",
    "thumbnail": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBAUEBAYFBQUGBgYHCQ4JCQgICRINDQoOFRIWFhUSFBQXGiEcFxgfGRQUHScdHyIjJSUlFhwpLCgkKyEkJST/2wBDAQYGBgkICREJCREkGBQYJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCT/wAARCAKjBLADASIA.....",
    "snippet": "Bad news for loyal long-time customers of T-Mobile: prices are about to go up by quite a lot.",
    "url": "https://tmo.report/2026/06/breaking-t-mobile-to-force-migrate-over-8-million-customers-to-more-expensive-plans",
    "success": true,
    "statusCode": 200,
    "langs": [
      "en"
    ],
    "acceptLanguage": "en",
    "htmlLang": "en-US",
    "text": "T-MobileNewsLeaks\nBreaking: T-Mobile To Force Migrate Over 8 Million Customers To More Expensive Plans\n JMAN100  JUNE 29, 2026 3 MIN READ \n\nWe covered news yesterday about certain rumblings at T-Mobile, specifically how many employees have been told to attend a meeting on Monday morning. We assumed it would be new plans to compete with Verizon.\n\nUnfortunately, we’ve learned a bit of exclusive news about what’s going to happen, and it isn’t the new plan extravaganza we thought it might be.\n\nForced Plan Migrations\n\nIn what can only be described as a company stabbing its long-time customers in the back, T-Mobile is apparently planning to force migrate customers off certain legacy plans.\n\nSpecifically, Simple Choice and ONE plan customers will be moved off of their grandfathered plans and onto more modern plans. Apparently, new plans are being created specifically for these customers. Magenta plan customers may also be affected, but it’s unclear right now.\n\nIt’s not fully clear exactly which plans will be migrated, but if we had to guess, it would be any plan not covered by the specific price lock that prevents price increases. The chart below explains the different versions of Price Lock and Un-Contract.\n\nSource: u/desterpot on Reddit\n\nIn addition, these changes are apparently hitting T-Mobile for Business customers as well.\n\nIt’s likely T-Mobile will try and spin the move as “good news” for these customers, possibly touting all the new benefits and perks you’ll get for just a small price increase.\n\nPrice Increase Details\n\nT-Mobile will also be increasing the cost of voice lines for these customers by $6 per line. Watch and tablet lines will increase by $3 per line. Customers with 5G Home Internet are also going to get a $6 price bump on those lines.\n\nThat would mean, for example, a ONE plan with 5 paid lines could see a $30 price increase. Free lines, presumably, will remain free, and not be subject to the $6 increase.\n\nThen there’s the unique plan perks. The popular Kickback promo, where customers on the ONE plan would get $10 off per line that stayed under 2GB of data use, is also ending.\n\nUsing the above example of a ONE plan with 5 paid lines, assuming each one was actively getting that credit, that could result in a price increase of $80 total. Outrageous.\n\nAs for the famous insider discount, theoretically that should migrate to the new plan just like it would if you had upgraded manually to a newer plan.\n\nNo “New” Plans After All\n\nIt turns out our initial guess of new plans was way off. T-Mobile has decided that, instead of making new cheaper plans to compete with Verizon and AT&T (and the number of cheaper MVNO plans), they’ll screw over long-time loyal customers instead.\n\nThis move can only be seen as T-Mobile shooting themselves in the foot. Customers barely tolerated the last time price increases hit legacy plans (and some didn’t tolerate it at all), so this time around it’s likely many customers will be shopping around for new cell plans.\n\nIt’s truly bewildering why T-Mobile would choose to do this at all, let alone a week after their biggest competitor announced a new super competitive price. It’s almost like they want customers to leave.\n\nThe (barely) good news is, apparently, T-Mobile won’t implement these changes for two weeks after announcing them, so you’ll have a little bit of time to decide what to do.\n\nWe’ll be publishing a new article in a few hours once all the details are public, so be sure to keep an eye out for that. For now, you might want to look into Verizon’s new Simplicity plan, or shop around for prepaid options instead. For now, though, RIP UnCarrier!\n\n  \n\n\nTags: Leaks News t-mobile price increase Plans Price Lock\nContinue Reading\nPrevious\nPrevious post:\nCould New Plans Be Coming To T-Mobile On Monday? [Updated]\nRELATED NEWS\nCould New Plans Be Coming To T-Mobile On Monday? [Updated]\nCould New Plans Be Coming To T-Mobile On Monday? [Updated]\n JUNE 28, 2026\nDiscord, Signal, And Others Now Work Via T-Mobile Satellite\nDiscord, Signal, And Others Now Work Via T-Mobile Satellite\n JUNE 26, 2026",
    "error": null,
    "extractor": "playwright",
    "published": "Mon, 29 Jun 2026 09:52:20 GMT",
    "googleNewsUrl": "https://news.google.com/rss/articles/CBMisAFBVV95cUxQRHlGeS1yT2RXSThBcUFnOGJLYm5haWt4QVNyQ3Y1S0RZNXptWkhHNzJ3ZWUydElXaFpvWnNZVnd2cERqalptb0NxMU82LVhTOHEwM2hpdHpvbmg5czhteFl1NDJ1akthWlVsQU5jNV9Oc2RSNjNiczBzdHlYMjNhdGxtX2EtSWt5UVpMZmphSTJGLWVYV2JXamdJSjJkV05vSVZIT3h1Qm8tN1lVNXZndA?oc=5",
    "query": null,
    "topic": "TECHNOLOGY",
    "topicRssUrl": "https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en&gl=US&ceid=US:en",
    "inputMode": "topic",
    "hl": "en",
    "gl": "US",
    "ceid": "US:en"
  },
  {
    "position": 2,
    "title": "Amid Steam Machine performance scrutiny, Valve removes '4K at 60FPS' claim - Eurogamer",
    "domain": "http://www.eurogamer.net",
    "thumbnail": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBAUEBAYFBQUGBgYHCQ4JCQgICRINDQoOFRIWFhUSFBQXGiEcFxgfGRQUHScdHyIjJSUlFhwpLCgkKyEkJST/2wBDAQYGBgkICREJCREkGBQYJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCT/wAARCAJ2BLADASIAAhEBAxEB/8QAHQAAAQUBAQEBAAAAAAAAAAAAAgABAwQFBgcICf/EAFkQAAEDAgQDBQUFAwgIBAMDDQEAAhEDIQQSMUEFUWEGEyJxgRQykaGxByNCwdEVUvAIM0NTYpLh8RYkJTRUcoKTc6KyszVEYxcmNnSEwhhFVWSjZYOUw9L/xAAaAQEBAQEBAQEAAAAAAAAAAAAAAQIDBAUG/8QAJxEBAQACAgICAgIDAQEBAAAAAAECEQMSITEEQRNRFDIFImEVcUL/2gAMAwEAAhEDEQA/APpQoSiKErAAoCpFGQmgGqEo3IXKASozdSFA5SqjPkhM6IibpiiwCYpymN1lYAodkaZRQEIDbyUiGCpVAfJBqFIR5dFk9o+0/CeymB9r4piRTkHu6IvUqkbN...",
    "snippet": "Valve appears to be walking back on the claims its expensive Steam Machine can consistently power native 4K60 gaming.",
    "url": "https://www.eurogamer.net/steam-machine-marketing-update-4k-60-fps-removed",
    "success": true,
    "statusCode": 200,
    "langs": [
      "en"
    ],
    "acceptLanguage": "en",
    "htmlLang": "en",
    "text": "Home  News\nAmid Steam Machine performance scrutiny, Valve removes '4K at 60FPS' claim\n\nWalk it back.\n\nImage credit: Eurogamer\nNews by Sherif Saed Contributor\nPublished on June 29, 2026\n 13 comments\n\nValve no longer claims the Steam Machine is capable of running games at 4K resolutions and 60 frames per second. The company updated the official Steam Machine product page to replace \"4K gaming at 60 FPS with FSR\", with \"Up to 4K gaming with FSR 4.1\".\n\nThe change was made without prior announcement, and was noticed by users on X and ResetEra forums. This is especially notable as, now that the Steam Machine has launched, its hardware capabilities are under increased scrutiny.\n\nSteam Machine: Official Overview and Quick Start Guide\nWatch on YouTube\n\nAs recently as February, Valve continued propagating the claim that 4K at 60FPS is a reasonable visual/performance target players can expect from the Steam Machine.\n\n\"In our testing the majority of Steam titles play great at 4K 60FPS with FSR on Steam Machine,\" Valve said on a Steam Machine FAQ. \"That said, there are some titles that currently require more upscaling than others, and it may be preferable to play at a lower framerate with VRR to maintain a 1080p internal resolution.\"\n\nIndeed, when media outlets reported on the Steam Machine's initial announcement, they likewise revealed, based on information provided by Valve, that 4K at 60FPS was the goal.\n\n\"Like the base model PS5 and Xbox Series X, the goal here is to support modern games with ray tracing at 4K resolution and 60 frames per second, using FSR upscaling (typically from a 1440p base resolution),\" Will Judd wrote in Eurogamer's Steam Machine preview in November last year.\n\nImage credit: Valve\n\nIt's not clear what prompted Valve to make that change, or when it was made. RPS deputy editor James Archer told us in a technical analysis Q&A that it's best to consider the Steam Machine as a 1080p device.\n\nADVERTISEMENT\n\n\"It's built for 1080p, to be sure, but with lowered settings and enough upscaling, you'd be surprised,\" said Archer, who added that major games will require using FSR upscaling on Performance mode to reach decent framerates.\n\nWe've reached out to Valve to try and understand why the change was made.\n\nAs Chris Tapsell notes in the Eurogamer Steam Machine review, the experience of living with the console-like PC is about a lot more than simple hardware specs. If you're the sort of person who's okay with the high price of the Steam Machine, you're not likely to be too fussed about its performance.\n\nLove Eurogamer.net? Make us a Preferred Source on Google and catch more of our coverage in your feeds.",
    "error": null,
    "extractor": "playwright",
    "published": "Mon, 29 Jun 2026 11:01:03 GMT",
    "googleNewsUrl": "https://news.google.com/rss/articles/CBMif0FVX3lxTFBLTUoxZUVrSFNwNURSZkR3Q0lXZmp4aE5SUGpSMGl0Nlhobzc1TE9IbTZRQ2FUdFY5ZmdYckdrNjIxeDltOVlCTjRWRmR2ZnM2LURMREh3cm9NQl9OdjhycGR6NDhiN1ZTZnJiemtzZ3I4eWxEbkRvNFV1bk5HM2c?oc=5",
    "query": null,
    "topic": "TECHNOLOGY",
    "topicRssUrl": "https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en&gl=US&ceid=US:en",
    "inputMode": "topic",
    "hl": "en",
    "gl": "US",
    "ceid": "US:en"
  }

Final failure result

{
  "url": "https://example.com",
  "success": false,
  "statusCode": null,
  "langs": ["it", "en"],
  "acceptLanguage": "it,en;q=0.9",
  "htmlLang": null,
  "text": null,
  "error": "HTTP fallback: ...",
  "extractor": null
}

Logs and performance metrics

The final actor includes structured logs that make it easier to debug and profile runs. These logs are designed to help verify each step of the actor and identify bottlenecks in RSS fetching, URL decoding, browser extraction, fallback extraction, and total runtime.

Notes and limitations

• Google News RSS is used only for discovery; final extraction runs against publisher pages, not Google-hosted article pages.
• maxSites applies to unique news sites per language/region pair, not to the total number of articles globally across all pairs.
• daysBack controls the search lookback window, and the final actor sets chunkDays = daysBack, so each edition is queried with a single time chunk covering the full requested range.
• daysBack applies to search-based discovery. Topic-based discovery reads the selected Google News topic feed and does not add after: or before: search filters.
• The extracted text depends on publisher HTML structure and on whether content is accessible to browser automation or plain HTTP requests.
• Some publishers may block automation, enforce paywalls, or deliver reduced content to automated clients.
• The languageAndRegion choices in the input schema can be represented as a practical whitelist, but Google does not publish a stable, official, exhaustive list of all supported ceid combinations.

Client code examples

Node.js

import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with API token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "\"samsung galaxy s25\" -ultra",
    "maxConcurrency": 10,
    "languageAndRegion": [
        "IT:it",
        "US:en",
        "LT:lt"
    ],
    "maxSites": 50,
    "daysBack": 7,
    "decodeInterval": 1
};

(async () => {
    // Run the Actor and wait for it to finish
    const run = await client.actor("nQxTzKe5yrbyzysYh").call(input);

    // Fetch and print Actor results from the run's dataset (if any)
    console.log('Results from dataset');
    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    items.forEach((item) => {
        console.dir(item);
    });
})();

Python

from apify_client import ApifyClient

# Initialize the ApifyClient with your API token
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "\"samsung galaxy s25\" -ultra",
    "maxConcurrency": 10,
    "languageAndRegion": [
        "IT:it",
        "US:en",
        "LT:lt",
    ],
    "maxSites": 50,
    "daysBack": 7,
    "decodeInterval": 1,
}

# Run the Actor and wait for it to finish
run = client.actor("nQxTzKe5yrbyzysYh").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

cURL

# Set API token
API_TOKEN=<YOUR_API_TOKEN>

# Prepare Actor input
cat > input.json <<'EOF'
{
  "searchQuery": "\"samsung galaxy s25\" -ultra",
  "maxConcurrency": 10,
  "languageAndRegion": [
    "IT:it",
    "US:en",
    "LT:lt"
  ],
  "maxSites": 50,
  "daysBack": 7,
  "decodeInterval": 1
}
EOF

# Run the Actor
curl "https://api.apify.com/v2/acts/nQxTzKe5yrbyzysYh/runs?token=$API_TOKEN" \
  -X POST \
  -d @input.json \
  -H 'Content-Type: application/json'