News Across the Web Scraper

News Across the Web Scraper

Scrape news articles from Semafor, Axios, Medium, The Intercept, The Verge, TechCrunch, and Wired in one run. Filter by search term and collect up to a million articles. Export to CSV, JSON, Excel, or XML.

News sites each have their own layout, feeds, and rate limits. This actor reads the public article feeds from seven major sources at once, filters them by a search term you provide, and returns every match in one fixed schema. No API keys, no per-site setup, no pagination code.

Who uses it	What they scrape News Across the Web for
Market researchers	Which topics a niche is discussing this week
Journalists	What competing outlets are covering on a story
PR teams	Where a brand or executive is mentioned across tech media
Data analysts	Building a clean dataset of recent articles for trend analysis

What it does

This Actor collects news articles from the selected sources, filters them by an optional search term, and returns each article as a flat row.

• 🔎 Search term filter: case-insensitive substring match on title and summary, so only relevant articles reach your dataset.
• 📰 Seven sources in one run: Semafor, Axios, Medium (Towards Data Science), The Intercept, The Verge, TechCrunch, and Wired.
• 📦 Flat row output: every article is one record with title, summary, source, URL, and publication date.
• ⚙️ Max articles control: set a hard cap from 1 to 1,000,000 per run to stay within your storage budget.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with News Across the Web data

📈 Track topics and trends.

A market researcher runs the actor weekly with a search term like 'generative AI' across all seven sources to see which angles are gaining traction.

🗞️ Monitor competitor coverage.

A journalist sets the search term to a breaking story and collects articles from The Verge, TechCrunch, and Wired to compare framing and sources.

📊 Build a media mention dataset.

A PR team runs the actor with their company name and exports the results to CSV for a monthly coverage report.

🧪 Feed a research pipeline.

A data analyst collects all articles from Medium (Towards Data Science) and filters by 'machine learning' to create a clean dataset for a literature review.

Why choose this scraper

	What you get
One schema for seven sites	No per-site parsing or cleanup, every article arrives in the same flat format
No API keys or logins	Reads public feeds directly, so you can start a run immediately
Search term filter built in	Only articles matching your term are returned, saving storage and processing time
Scales to a million articles	Set maxItems as high as you need for large-scale collection

How it compares

No other Store actor targets News Across the Web the same way, so the honest comparison is with the alternatives teams actually weigh.

	News Across the Web Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When News Across the Web changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor from a search term and a list of sources, and the filter runs as each article is read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10
}

A larger pull:

{
 "maxItems": 200
}

Pricing

Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$2.10
1,000 results	$21.00
10,000 results	$210.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the News Across the Web Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to News Across the Web through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/news-across-the-web-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your search term is spelled correctly and is not too specific. Also verify that at least one source is selected in the sources array. If the term is very niche, try broadening it or removing it entirely.

Why are some sources returning fewer articles than others?

Each source has a different feed size and update frequency. Some sources may only expose a limited number of recent articles in their public feeds.

Why did my run stop before reaching maxItems?

The actor stops when it has read all available articles from the selected sources. If the total number of articles across those sources is less than maxItems, the run ends early.

Can I get the full text of an article?

No, this actor only returns the title and summary from the source feeds. To get full text, you would need a different actor that fetches individual article pages.

Why is my search term not matching articles I can see on the site?

The filter is applied to the title and summary fields only. If the term appears only in the article body or in metadata not included in the feed, it will not match.

FAQ

Question	Answer
Which news sources does this actor scrape?	It scrapes Semafor, Axios, Medium (Towards Data Science), The Intercept, The Verge, TechCrunch, and Wired. You can select any subset of these sources in the input.
Do I need API keys or logins for these sites?	No. The actor reads the public article feeds directly, so no API keys, OAuth, or account registration is required.
How does the search term filter work?	The query input is a case-insensitive substring filter applied to the article title and summary. Only articles containing the term are returned.
Can I scrape all articles from a source without a search term?	Yes. Leave the query field empty and the actor will return all articles it can find from the selected sources, up to the maxItems limit.
What is the maximum number of articles I can collect?	You can set maxItems up to 1,000,000 articles per run. The default is 10.
What format is the output data in?	Each article is returned as a flat row with fields like title, summary, source, URL, and publication date. You can export the dataset to CSV, JSON, Excel, or XML.
Does this actor scrape full article text?	No, it returns the title and summary as shown in the source feeds. It does not fetch the full body text of each article.
Can I schedule this actor to run automatically?	Yes, you can schedule runs on Apify with any interval, such as hourly or daily, to keep your dataset up to date.
Is this actor free to use?	The actor itself is free to run on the Apify platform. You only pay for the platform compute units you consume, and the free tier includes a monthly allowance.
Can I filter by publication date?	No, the current input schema does not include a date filter. The actor returns the most recent articles available from each source feed.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by News Across the Web. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.