Article & Main Content Extractor (Web Page to Markdown) avatar

Article & Main Content Extractor (Web Page to Markdown)

Pricing

from $0.30 / 1,000 pages

Go to Apify Store
Article & Main Content Extractor (Web Page to Markdown)

Article & Main Content Extractor (Web Page to Markdown)

Get the readable main content of any web page as clean Markdown or text, with title, author, date and word count. Navigation, ads and boilerplate removed. Ideal for LLMs and RAG. Failures free. $0.50 per 1,000 pages.

Pricing

from $0.30 / 1,000 pages

Rating

0.0

(0)

Developer

Aaron Rabinowe

Aaron Rabinowe

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

4 days ago

Last modified

Features

Limited permissions

Share

Article & Main Content Extractor: any web page to clean Markdown

Get just the readable content of web pages (the article, post or doc) as clean Markdown or plain text, for $0.50 per 1,000 pages. Navigation menus, ads, footers, cookie notices and sidebars are stripped, and headings, lists, bold text and tables are kept. Each result includes the title, author, publish date, site name, description and word count. Failed URLs are never charged.

Example: Wikipedia's "Web scraping" article comes back as about 4,000 words of clean Markdown, and a Cloudflare blog post as about 1,600.

What people use it for

  • LLMs, RAG & AI agents: clean, token-efficient text for embeddings, summarization and question answering, without the menus and boilerplate that waste context.
  • Content monitoring & research: pull the text of articles, blog posts and docs for analysis.
  • Archiving & reading tools: save readable copies of pages.
  • Datasets: build text corpora from lists of URLs.

Input

{ "urls": ["https://en.wikipedia.org/wiki/Web_scraping", "https://docs.python.org/3/tutorial/introduction.html"], "outputFormat": "markdown", "includeLinks": false, "maxPages": 500 }

Output

url, finalUrl, httpStatus, title, author, date, siteName, description, language, content, format, wordCount, scrapedAt. URLs that fail (blocked, not HTML, no readable content) are listed with the reason in the SUMMARY record.

Pricing

$0.50 per 1,000 pages ($0.0005 each) on the free plan, cheaper on larger Apify plans. Only successful extractions are charged.

Good to know

  • Built for content pages. Articles, blogs, docs and news extract best. Homepages and app-like pages have little "main content".
  • No JavaScript rendering. Fast and cheap; pages that need JavaScript to show their text are reported as having no readable content.
  • Honest crawler. It identifies itself as a bot (many sites, including Wikipedia, block disguised crawlers) and only fetches the URLs you give it.

Frequently asked questions

Why Markdown?

Markdown keeps headings, lists and links while dropping menus, ads and footers, which makes it ideal input for LLMs, RAG and summarization.

Does it get the author and date?

Yes, when the page publishes them: title, author, publish date and word count are returned with the content.

Does it work on JavaScript-heavy sites?

It reads the server-rendered HTML, which works for most news sites, blogs and documentation. Pages that render only in the browser may return less text.

How do I run it on a schedule?

In Apify Console, open the Actor and click Schedule: hourly, daily or any cron expression. Each run's results land in a new dataset you can export as CSV, Excel or JSON.

Can I call it from my own code or no-code tools?

Yes. Use the Apify API or the official Python and JavaScript clients, or connect it to Make, Zapier, n8n, Google Sheets or a webhook through Apify's built-in integrations.

Is there a free trial?

Apify's free plan includes monthly platform credit, enough to test this Actor on your real data before paying anything.

More tools from the same developer

Questions or a feature request? Open an issue on the Actor's Issues tab and I'll usually reply within a day.