Web Scraper to Markdown: Page & PDF Content for LLMs

What it does: Turn web pages and PDFs into clean, LLM-ready markdown or text with title, author and date; respects robots.txt.

Who it's for: RAG pipelines, research and browsing agents, summarisers and knowledge-base builders.

Price: Pay per event: $0.005 per page (a PDF is billed per page extracted, capped by maxPdfPages). Failed URLs are not charged. You only pay Apify for what you run; there is no monthly fee for this Actor.

Give it URLs, get back clean, LLM-ready markdown or plain text:

• Web pages: main content only (menus, cookie banners, ads and footers removed), headings, lists and tables kept, plus title, author, date, site name, language.
• PDFs: text extracted page by page (with page markers), PDF metadata, page count, and a warning when a PDF looks scanned (image only, no text layer).
• Follows redirects, detects PDFs by content, respects robots.txt, 40 MB file limit.

Input

{"urls": ["https://en.wikipedia.org/wiki/Markdown", "https://arxiv.org/pdf/1706.03762"], "format": "markdown", "maxPdfPages": 50}

Output (one row per URL)

url, final_url, type (html/pdf), title, content, words, chars, metadata, pages_total, pages_extracted, truncated, billed_pages.

Pricing

Pay per event $0.005 per page: one web page = 1 page; a PDF is billed per page extracted (capped by maxPdfPages). Failed URLs are not charged.

Notes

Only fetches public pages; it does not log in, bypass paywalls or captchas. You are responsible for having the right to use the content you convert. Built with trafilatura (Apache 2.0) and pypdf (BSD).

Quick example

Input

{
 "urls": [
  "https://en.wikipedia.org/wiki/Markdown"
 ],
 "format": "markdown",
 "maxPdfPages": 50,
 "includeLinks": false,
 "maxChars": 200000
}

Output (one dataset item, shortened)

{
 "url": "https://en.wikipedia.org/wiki/Markdown",
 "final_url": "https://en.wikipedia.org/wiki/Markdown",
 "format": "markdown",
 "type": "html",
 "title": "Markdown - Wikipedia",
 "content": "# Markdown\n\n\n| Markdown |  | \n|---|---|\n| Filename extensions | `.md`, `.markdown`<sup>\\[1\\]</sup><sup>\\[2\\]</sup> | \n| Internet media type | `text/markdown`<su...",
 "metadata": {
  "title": "Markdown - Wikipedia",
  "author": "Authority control databases International FAST National United States Israel",
  "date": "2005-08-09",
  "sitename": "Wikimedia Foundation, Inc."
 },
 "chars": 21325,
 "words": 3001,
 "truncated": false,
 "billed_pages": 1,
 "error": null
}

Use it from AI agents (MCP) and code

• Apify MCP server: add https://mcp.apify.com/?tools=tudorlane/page-to-markdown to Claude, Cursor, ChatGPT connectors, VS Code or any MCP client (sign in with Apify OAuth or an API token). The Actor appears as a tool the model can call.
• REST (sync): POST https://api.apify.com/v2/acts/tudorlane~page-to-markdown/run-sync-get-dataset-items?token=YOUR_TOKEN with the input JSON as the body; results come back in the response.
• SDKs / no-code: apify-client (Python/JS), LangChain, LlamaIndex, CrewAI, n8n, Make and Zapier integrations.

FAQ

JavaScript-heavy sites? It fetches server HTML; pages that render only in the browser may return little text.

Paywalls/logins? Not bypassed, by design.

Scanned PDFs? Flagged with a warning (no OCR).

What if the source is down? You get an error row and are not charged for it.

Support? Open an issue on the Actor's Issues tab.

---

Keywords: web page to markdown API, HTML to markdown, PDF to text API, LLM-ready text, RAG ingestion, web content extraction, MCP tool, Apify Actor, AI agent tool.

Search terms: web scraper, markdown converter, HTML to markdown, website content for LLM, RAG ingestion, PDF to markdown (single URLs; not a full-site crawler)