Web Page to Markdown & Text: clean main content from any URL

The main content of web pages as Markdown and plain text, up to 500 URLs per run: title, description, author, date, language, word count, links and images, without menus, cookie banners and footers, ready for LLMs, search indexes and archives. Pay per page.

Why this scraper

• Only delivered results are charged. Inputs that fail come back as clear error records at no cost.
• No browser: the page is fetched once and cleaned with trafilatura, the boilerplate remover used to build research corpora. Measured on the platform: the Wikipedia article on web scraping (4,000 words, 263 links) and Apify's pricing page in a few seconds for four hundredths of a cent.
• Main content only: menus, cookie banners, sidebars and footers are gone, the article or product text stays, as Markdown (headings, links, tables, images) and as plain text in the same row. The page's own metadata rides along: title, description, author, date, language, site name, canonical URL, plus word, link and image counts. Every URL is checked against its host's robots.txt first, and a page that refuses or holds no readable text is a free error row.

Who this is for

Paste page URLs in urls (up to 500 per run). Keep includeLinks and includeTables on for Markdown with links and tables, or switch them off for cleaner prose. Switch on fullPage to keep everything readable on the page instead of the main content only. Every row carries the URL and final URL, title, description, author, date, language, site name, the Markdown, the plain text, word, link and image counts, the canonical URL and the fetch time.

Who this is not for

This reads the HTML the server sends. A page that builds its text with JavaScript after loading (many app-like sites, some news front pages) comes back with little or nothing and is reported as NO_CONTENT, free of charge; a site that blocks datacenter traffic answers 403 and shows as BLOCKED. Main-content detection is a heuristic: on pages without a clear article (home pages, listings) it keeps what looks like body text and may drop teasers; use fullPage there. Author and date are only as good as the page's own tags.

Input example

{
    "urls": [
        "https://en.wikipedia.org/wiki/Web_scraping"
    ],
    "includeLinks": true,
    "includeTables": true,
    "fullPage": false
}

Output example

• url
• finalUrl
• title
• description
• author
• publishedDate
• language
• siteName
• markdown
• text
• wordCount
• linkCount
• imageCount
• canonicalUrl
• fetchedAt

Error codes: INVALID_URL, ROBOTS_DISALLOWED, NO_CONTENT, FETCH_FAILED, BLOCKED.

One delivered row looks like this:

{
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "finalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "title": "Web scraping - Wikipedia",
  "description": null,
  "author": null,
  "publishedDate": "2005-09-17",
  "language": "en",
  "siteName": "Wikimedia Foundation, Inc.",
  "markdown": "**Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](http...",
  "text": "Web scraping, web harvesting, or web data extraction is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](https://en.wikip...",
  "wordCount": 4008,
  "linkCount": 263,
  "imageCount": 0,
  "canonicalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "fetchedAt": "2026-09-16T18:26:17Z",
  "status": "ok"
}

Related actors from steadydata

• structured-data-extractor (publishing soon): the same page's JSON-LD, Open Graph and meta tags
• sitemap-urls: every page of a site to convert
• pdf-text-extractor: the same for PDF links
• technical-seo-audit: the technical checks of the same pages
• Used together with sitemap-urls and pdf-text-extractor in a recipe: Feed a RAG pipeline with clean page text.

Pricing

Pay per event: one page-converted event per delivered result. No charge for inputs that fail, no separate platform-usage surcharge.

Free Apify plan: this actor delivers up to 25 rows per run for accounts on the Apify free plan, and then stops with a message. That limit is set by us, not by Apify. It exists so the actor keeps paying for itself for the people who do pay. Any paid Apify plan runs it at full size, billed per delivered row, with failed rows never charged.

Reviews: if this actor saves you time, a short review on this page is the one thing that helps most. Ratings are what other buyers look at first, and we have no other way to ask.

FAQ

Is this good input for an LLM? That is the point: clean Markdown without navigation noise, one row per page, with the metadata to cite it.

Does it render JavaScript? No. Server-rendered pages, articles, documentation, blogs and most product pages work; single-page apps do not.

Can I convert a whole site? Feed it the output of the Sitemap URLs actor below, 500 pages per run.

What about PDFs? Use the PDF Text Extractor below; this actor delivers a free error row for non-HTML content.

Do I pay for pages that fail? No. Only pages delivered with readable content are charged.

Is personal data collected? wordCount counts words in the plain text; linkCount and imageCount count what the Markdown kept; publishedDate is the date the page declares, as YYYY-MM-DD.

What happens when the source changes? Sources change from time to time; that is the nature of this work. The actor is monitored daily and fixed fast, and while it is broken you are not charged, because only delivered results cost anything.