Web Page to Markdown & Text: clean main content from any URL

The main content of web pages as Markdown and plain text, up to 500 URLs per run: title, description, author, date, language, word count, links and images, without menus, cookie banners and footers, ready for LLMs, search indexes and archives. Pay per page.

Why this scraper

• Only delivered results are charged. Inputs that fail come back as clear error records at no cost.
• No browser: the page is fetched once and cleaned with trafilatura, the boilerplate remover used to build research corpora. Measured on the platform: the Wikipedia article on web scraping (4,000 words, 263 links) and Apify's pricing page in a few seconds for four hundredths of a cent.
• Main content only: menus, cookie banners, sidebars and footers are gone, the article or product text stays, as Markdown (headings, links, tables, images) and as plain text in the same row. The page's own metadata rides along: title, description, author, date, language, site name, canonical URL, plus word, link and image counts. Every URL is checked against its host's robots.txt first, and a page that refuses or holds no readable text is a free error row.

Who this is for

Paste page URLs in urls (up to 500 per run). Keep includeLinks and includeTables on for Markdown with links and tables, or switch them off for cleaner prose. Switch on fullPage to keep everything readable on the page instead of the main content only. Every row carries the URL and final URL, title, description, author, date, language, site name, the Markdown, the plain text, word, link and image counts, the canonical URL and the fetch time.

Who this is not for

This reads the HTML the server sends. A page that builds its text with JavaScript after loading (many app-like sites, some news front pages) comes back with little or nothing and is reported as NO_CONTENT, free of charge; a site that blocks datacenter traffic answers 403 and shows as BLOCKED. Main-content detection is a heuristic: on pages without a clear article (home pages, listings) it keeps what looks like body text and may drop teasers; use fullPage there. Author and date are only as good as the page's own tags.

Input fields

Field	Type	Required or default	What it does
urls	list of text	required	One URL per row, up to 500.
includeLinks	true/false	true	On: links stay as url in the Markdown. Off: link text only.
includeTables	true/false	true	Tables become Markdown tables.
fullPage	true/false	false	Off: only the main content (article, product text) is extracted. On: everything readable on the page, menus and footers included.

Input example

{
    "urls": [
        "https://en.wikipedia.org/wiki/Web_scraping"
    ],
    "includeLinks": true,
    "includeTables": true,
    "fullPage": false
}

Output example

Field	Type	What it holds	Example
url	text	The page or file this row is about.	https://en.wikipedia.org/wiki/Web_scraping
finalUrl	text	The page that was really read, after every redirect.	https://en.wikipedia.org/wiki/Web_scraping
title	text	The title as it stands in the source.	Web scraping - Wikipedia
description	text	The page's own meta description.	Thousands of tools to automate your business. Get real-ti...
author	text	The author the source names; empty when it names none.
publishedDate	text	The publication date the page states; empty when it states none.	2005-09-17
language	text	The language of the text, as a language code.	en
siteName	text	The site the page belongs to, as the page names itself.	Wikimedia Foundation, Inc.
markdown	text	The article as Markdown, with headings, lists and links kept intact; menus, cookie bars and footers left out.	**Web scraping**, **web harvesting**, or **web data extra...
text	text	The same article as plain text, without any markup.	Web scraping, web harvesting, or web data extraction is [...
wordCount	number	How many words the article holds, the quickest measure of how substantial a page is.	3943
linkCount	number	How many links stand in the article itself.	243
imageCount	number	How many images the page carries.	0
canonicalUrl	text	The address the page names as its own canonical one, which can differ from the one you asked for.	https://en.wikipedia.org/wiki/Web_scraping
fetchedAt	text	When the page was read, in UTC.	2026-09-30T06:51:00Z

Error codes: INVALID_URL, ROBOTS_DISALLOWED, NO_CONTENT, FETCH_FAILED, BLOCKED.

One delivered row looks like this:

{
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "finalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "title": "Web scraping - Wikipedia",
  "description": null,
  "author": null,
  "publishedDate": "2005-09-17",
  "language": "en",
  "siteName": "Wikimedia Foundation, Inc.",
  "markdown": "**Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](http...",
  "text": "Web scraping, web harvesting, or web data extraction is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](https://en.wikip...",
  "wordCount": 4008,
  "linkCount": 263,
  "imageCount": 0,
  "canonicalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "fetchedAt": "2026-09-16T18:26:17Z",
  "status": "ok"
}

Related actors from steadydata

• structured-data-extractor: the same page's JSON-LD, Open Graph and meta tags
• sitemap-urls: every page of a site to convert
• pdf-text-extractor: the same for PDF links
• technical-seo-audit: the technical checks of the same pages
• Used together with sitemap-urls and pdf-text-extractor in a recipe: Feed a RAG pipeline with clean page text.

Pricing

Pay per event: one page-converted event per delivered result. No charge for inputs that fail, no separate platform-usage surcharge.

Free Apify plan: this actor delivers up to 25 rows per run for accounts on the Apify free plan, and then stops with a message. That limit is set by us, not by Apify. It exists so the actor keeps paying for itself for the people who do pay. Any paid Apify plan runs it at full size, billed per delivered row, with failed rows never charged.

Reviews: if this actor saves you time, a short review on this page is the one thing that helps most. Ratings are what other buyers look at first, and we have no other way to ask.

FAQ

Is this good input for an LLM? That is the point: clean Markdown without navigation noise, one row per page, with the metadata to cite it.

Does it render JavaScript? No. Server-rendered pages, articles, documentation, blogs and most product pages work; single-page apps do not.

Can I convert a whole site? Feed it the output of the Sitemap URLs actor below, 500 pages per run.

What about PDFs? Use the PDF Text Extractor below; this actor delivers a free error row for non-HTML content.

Do I pay for pages that fail? No. Only pages delivered with readable content are charged.

Is personal data collected? wordCount counts words in the plain text; linkCount and imageCount count what the Markdown kept; publishedDate is the date the page declares, as YYYY-MM-DD.

What happens when the source changes? Sources change from time to time; that is the nature of this work. The actor is monitored daily and fixed fast, and while it is broken you are not charged, because only delivered results cost anything.

Can I call it from my own code, like an API? Yes. Every run is available through the Apify API: start it with one HTTP request, then read the results as JSON, CSV or Excel from the dataset endpoint, or use the official Python and JavaScript clients. The input is the same JSON as in the input example above, so a scheduled script, a Make or Zapier step or an AI agent can call it without any changes.