Article Extractor – News & Blog Articles to JSON & Markdown

Extract news articles and blog posts to clean, structured JSON: title, author, published and modified date, lead image, language, tags, section, clean text and Markdown, plus word count, reading time and canonical URL. Paste article URLs, or just a news site or blog URL and the articles are found for you through its RSS/Atom feeds, Google News sitemaps, XML sitemaps and section pages. Add a date filter or the only-new-articles mode to get a scheduled news feed for media monitoring, AI/RAG pipelines and content research.

No browser, so it is fast and costs $0.30 per 1,000 articles. Pages that turn out not to be articles are free.

What it does

• Two inputs, mixed freely.
  • Article URLs are extracted directly.
  • Site or section URLs (https://github.blog, https://www.bbc.com/news, techcrunch.com) are crawled for their newest articles:
    1. RSS / Atom / JSON feeds (<link rel="alternate">, /feed, /rss.xml, ...), newest first;
    2. Google News sitemaps and XML sitemaps listed in robots.txt (gzip and sitemap indexes supported; for a section URL like /news the matching sitemap is preferred);
    3. article-looking links on the start page.
• Article detection. Every discovered page is checked: schema.org NewsArticle / BlogPosting / Article JSON-LD, og:type=article, a publish date, enough words, and not a headline list. Home, tag, category, author and other listing pages are skipped and not charged.
• Rich metadata from JSON-LD (including @graph and author references), OpenGraph / article:* tags, Parse.ly and Dublin Core meta tags, <time> elements, bylines and author links, with the date in the URL as the last fallback (dateSource tells you where the date came from).
• Clean body. The article body container is located (itemprop=articleBody, .entry-content, <article> ...); navigation, ads, share bars, cookie banners and "Most popular / Related / Read more" blocks are removed. Output as plain text and as markdown (headings, lists, tables, code, quotes, optional links and images).
• Date filter (publishedAfter: 2026-10-01 or 7 days). Old entries in feeds and sitemaps are dropped before downloading, so they cost nothing.
• Only-new mode for scheduled runs. Processed URLs are remembered in a named key-value store; the next run extracts only articles that were not seen before.
• Per-URL errors instead of crashes. 404s, PDFs, robots.txt blocks, JavaScript-only pages and paywalled articles become rows with a status and an error message. They are free.
• Polite crawling. Honest User-Agent with a contact address, robots.txt and Crawl-delay respected, 2 parallel requests per site by default, retries with backoff on 429/5xx.

Use cases

• AI / RAG / LLM pipelines: feed clean Markdown with titles, dates and authors into a vector database or an LLM summarizer.
• Media monitoring and news alerts: schedule hourly with onlyNew and send new articles to Slack, email, Google Sheets or a webhook.
• Content and SEO research: publishing frequency, word counts, authors, tags and sections of competitor blogs.
• Datasets: build a news or blog corpus with consistent metadata for classification, sentiment or trend analysis.
• Newsletters and digests: collect this week's articles from a list of publishers in one table.

Input example

{
  "startUrls": [
    { "url": "https://github.blog" },
    { "url": "https://www.bbc.com/news" },
    { "url": "https://blog.cloudflare.com/post-quantum-geo-key-manager/" }
  ],
  "maxArticlesPerSite": 10,
  "publishedAfter": "3 days",
  "onlyNew": false
}

Field	Default	Meaning
startUrls	–	Article URLs and/or site / section URLs
mode	auto	auto: article URLs are extracted, other URLs are crawled as sites. site: always discover. urls: extract exactly these URLs, no discovery
maxArticlesPerSite	20	Articles per site URL (newest first when dates are known)
maxArticles	0	Cap for the whole run (0 = none)
publishedAfter	–	Absolute or relative date
onlyNew / stateKey	false	Monitoring memory between runs
useFeeds / useSitemaps / useLinks	true	Discovery sources for site URLs
minWords	150	Discovered pages with fewer words are not articles
includeText / includeMarkdown	true	Which body formats to return

Output example

One row per article (text and Markdown shortened):

{
  "url": "https://techcrunch.com/2026/10/25/grok-releases-the-first-seven-minutes-of-its-ai-generated-odyssey/",
  "canonicalUrl": "https://techcrunch.com/2026/10/25/grok-releases-the-first-seven-minutes-of-its-ai-generated-odyssey/",
  "status": "ok",
  "title": "Grok releases the first seven minutes of its AI-generated ‘Odyssey’",
  "description": "The footage looks okay, as long as you're not seeking artistry, inspiration, or human interest.",
  "author": "Anthony Ha",
  "authors": ["Anthony Ha"],
  "publishedAt": "2026-10-25T21:31:16Z",
  "modifiedAt": "2026-10-26T02:24:45Z",
  "dateSource": "json-ld",
  "leadImage": "https://techcrunch.com/wp-content/uploads/2026/03/grok-getty.jpg",
  "language": "en",
  "siteName": "TechCrunch",
  "section": "AI",
  "tags": ["Elon Musk", "SpaceX", "xAI"],
  "wordCount": 540,
  "readingTimeMinutes": 3,
  "text": "Grok releases the first seven minutes of its AI-generated ‘Odyssey’\n\nThree months after Elon Musk declared ...",
  "markdown": "# Grok releases the first seven minutes of its AI-generated ‘Odyssey’\n\nThree months after Elon Musk [declared](https://x.com/...) ...",
  "isArticle": true,
  "articleType": "NewsArticle",
  "detection": ["json-ld:NewsArticle", "og:type=article", "date:json-ld", "words:540"],
  "discoveredVia": "feed",
  "site": "techcrunch.com",
  "httpStatus": 200,
  "contentHash": "a8a01344a9377446…",
  "fetchedAt": "2026-10-26T11:59:09Z"
}

Rows that are not extracted articles have a status such as failed, blocked, robotsBlocked, skipped (not HTML), empty, needsBrowser, paywalled, noArticlesFound or invalidUrl, and an error text. The OUTPUT record in the key-value store has a per-start-URL summary: feeds and sitemaps used, candidates, pages fetched, articles, non-article pages, date-filtered and already-seen counts.

Dataset views: Articles (table with lead image), Text & Markdown, Metadata.

Pricing

Pay per event. You pay only for extracted articles.

Event	Price
Article extracted	$0.0003 ($0.30 / 1,000)
Actor start	$0.00005

Free: non-article pages found during discovery, pages older than your date filter, already-seen URLs, errors, blocked or paywalled pages.

Compared with other Store Actors (public Store prices, 2026-10-26):

Actor	What you pay per 1,000 articles
Article Extractor (this Actor)	$0.30, no compute charges
tropical_prune/article-news-extractor	$1.00
automation-lab/news-article-extractor	$2.30 + $0.005 per run
lukaskrivka/article-extractor-smart	Free Actor, but you pay the platform compute of a browser-capable crawler

Set Maximum cost per run in the run options to cap spending. The Actor stops cleanly when the limit is reached and never fetches more articles than the budget allows.

FAQ

Which sites work? Server-rendered news sites, magazines, company blogs, WordPress, Ghost, Substack-style and docs blogs: most of the web's articles. Pages that need JavaScript to show their text return needsBrowser and are not charged.

How are the newest articles chosen? Feed entries and Google News sitemap entries carry dates, so they are sorted newest first. Regular sitemaps are ordered by lastmod. Links from the start page come next. With publishedAfter, entries known to be older are dropped before they are downloaded.

What about paywalls? The Actor does not bypass paywalls or logins. Articles that declare isAccessibleForFree: false are returned as paywalled rows (free) without text. Metered sites may return only the public teaser.

Which sites are refused? Medium (including Medium custom domains), Reddit, Google News and social networks (X/Twitter, Facebook, Instagram, LinkedIn, TikTok, YouTube, Pinterest, Threads, Quora), because their terms do not allow automated extraction. Such URLs return a blocked row and are not charged.

How does only-new mode work? URLs that were extracted, skipped as non-articles or filtered by date are stored in a named key-value store (article-extractor-…, derived from your start URLs or from stateKey). On the first run you get the newest maxArticlesPerSite articles. The rest of that run's feed and sitemap entries are also marked as seen, so later runs return only articles published since, without back-filling the archive.

Article URL mode. With mode: "urls" every URL you give is extracted as-is and charged when it has readable text, even if it does not look like an article (isArticle tells you what the detector thinks).

Is it legal? The Actor only reads pages you point it to, identifies itself, and respects robots.txt and crawl delays. You are responsible for how you use the content (copyright, the site's terms, personal data in bylines). For republishing, get the publisher's permission.

Limits

• No JavaScript rendering, logins, CAPTCHAs or anti-bot bypass. Sites behind a Cloudflare challenge return failed (HTTP 403).
• Language is taken from the page (lang, JSON-LD, og:locale), not detected from the text.
• Author detection relies on markup; a few sites only show the author in images or scripts.
• Discovery reads at most 10 sitemap files per site and a page budget of 3 × maxArticlesPerSite + 10 candidates (maxPagesPerSite to change it).

Use with AI agents / Apify MCP

• Apify MCP server: add gazidev/article-extractor to your MCP client (Claude Desktop, Cursor, VS Code) via https://mcp.apify.com?actors=gazidev/article-extractor. An agent can call it with {"startUrls":["https://www.bbc.com/news"],"maxArticlesPerSite":5,"publishedAfter":"1 day"} and read today's articles as Markdown.
• API: POST https://api.apify.com/v2/acts/gazidev~article-extractor/run-sync-get-dataset-items?token=... with the input JSON returns the articles directly. It works as a "read this article" tool for LangChain, LlamaIndex or n8n.
• Scheduled news feed: create a Task with onlyNew: true, schedule it, and connect the Slack, Google Sheets or webhook integration.

More from the website toolkit

• RSS Feed Reader: any RSS, Atom or JSON Feed to JSON, OPML import, only-new items.
• Website to Markdown: any URL, sitemap or whole site to Markdown with RAG chunks and llms.txt.
• Sitemap Extractor: every URL from sitemap.xml, with status checks and only-new mode.