Article & Main Content Extractor (Web Page to Markdown)
Pricing
from $0.30 / 1,000 pages
Article & Main Content Extractor (Web Page to Markdown)
Get the readable main content of any web page as clean Markdown or text, with title, author, date and word count. Navigation, ads and boilerplate removed. Ideal for LLMs and RAG. Failures free. $0.50 per 1,000 pages.
Pricing
from $0.30 / 1,000 pages
Rating
0.0
(0)
Developer
Aaron Rabinowe
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Categories
Features
Share
Article & Main Content Extractor: any web page to clean Markdown
Get just the readable content of web pages (the article, post or doc) as clean Markdown or plain text, for $0.50 per 1,000 pages. Navigation menus, ads, footers, cookie notices and sidebars are stripped, and headings, lists, bold text and tables are kept. Each result includes the title, author, publish date, site name, description and word count. Failed URLs are never charged.
Example: Wikipedia's "Web scraping" article comes back as about 4,000 words of clean Markdown, and a Cloudflare blog post as about 1,600.
What people use it for
- LLMs, RAG & AI agents: clean, token-efficient text for embeddings, summarization and question answering, without the menus and boilerplate that waste context.
- Content monitoring & research: pull the text of articles, blog posts and docs for analysis.
- Archiving & reading tools: save readable copies of pages.
- Datasets: build text corpora from lists of URLs.
Input
{ "urls": ["https://en.wikipedia.org/wiki/Web_scraping", "https://docs.python.org/3/tutorial/introduction.html"], "outputFormat": "markdown", "includeLinks": false, "maxPages": 500 }
Output
url, finalUrl, httpStatus, title, author, date, siteName, description, language, content, format, wordCount, scrapedAt. URLs that fail (blocked, not HTML, no readable content) are listed with the reason in the SUMMARY record.
Pricing
$0.50 per 1,000 pages ($0.0005 each) on the free plan, cheaper on larger Apify plans. Only successful extractions are charged.
Good to know
- Built for content pages. Articles, blogs, docs and news extract best. Homepages and app-like pages have little "main content".
- No JavaScript rendering. Fast and cheap; pages that need JavaScript to show their text are reported as having no readable content.
- Honest crawler. It identifies itself as a bot (many sites, including Wikipedia, block disguised crawlers) and only fetches the URLs you give it.
Frequently asked questions
Why Markdown?
Markdown keeps headings, lists and links while dropping menus, ads and footers, which makes it ideal input for LLMs, RAG and summarization.
Does it get the author and date?
Yes, when the page publishes them: title, author, publish date and word count are returned with the content.
Does it work on JavaScript-heavy sites?
It reads the server-rendered HTML, which works for most news sites, blogs and documentation. Pages that render only in the browser may return less text.
How do I run it on a schedule?
In Apify Console, open the Actor and click Schedule: hourly, daily or any cron expression. Each run's results land in a new dataset you can export as CSV, Excel or JSON.
Can I call it from my own code or no-code tools?
Yes. Use the Apify API or the official Python and JavaScript clients, or connect it to Make, Zapier, n8n, Google Sheets or a webhook through Apify's built-in integrations.
Is there a free trial?
Apify's free plan includes monthly platform credit, enough to test this Actor on your real data before paying anything.
More tools from the same developer
- Ats Jobs Scraper: every open job from a company's careers page (Workday, Greenhouse, Lever, Ashby and more)
- Tech Job Search: search 4,400+ tech and startup career pages by keyword
- Remote Jobs Search: remote roles straight from company career pages
- Job Salary Data: real salary ranges from live job postings
- Hiring Companies Finder: companies hiring for a role right now (sales hiring signals)
- Website Contact Extractor: emails, phones and social profiles from a list of websites
- Domain Intelligence: WHOIS/RDAP dates, DNS, email provider and tech stack for domains
- Sitemap Url Extractor: every URL a website lists in its sitemaps
- Website Screenshot: full-page and mobile screenshots without cookie banners
- Pdf Text Extractor: text from PDFs at any URL, page by page
- Rss Feed Reader: any RSS or Atom feed as clean JSON
Questions or a feature request? Open an issue on the Actor's Issues tab and I'll usually reply within a day.