Web Page to Markdown & Link Preview: clean Markdown for LLMs plus Open Graph, Twitter card and favicons

Web Page to Markdown & Link Preview is an Apify Actor that turns a list of URLs into one dataset row per page: the page's main content as clean Markdown (for ChatGPT, Claude, agents and RAG pipelines) and its link preview metadata (title, description, Open Graph and Twitter card images, favicons, author and dates). It fetches without a headless browser, refuses private-network and cloud-metadata addresses, honours robots.txt for Markdown, and you pay only for pages that return a result ($0.0015 each, or $1.50 per 1,000).

What does Web Page to Markdown & Link Preview do?

You give it a list of public URLs and pick a mode:

• both (default): one row with the preview and the Markdown, at the price of one page.
• preview: link preview only, the same data Slack or Discord show when you paste a link.
• markdown: the main content only, as Markdown.

For each URL it returns:

• Preview: title, description, site_name, canonical, lang, type, image and images[] (absolute URLs, with width, height and alt when declared), videos[], favicons[], theme_color, author, published_time, modified_time, keywords, oembed and twitter card fields. It falls back from Open Graph to Twitter cards to plain HTML (<title>, meta description, first paragraph, first real image), so pages without social tags still give a usable card.
• Markdown: the article, docs page or README without menus, footers, cookie banners or ads, plus word_count, reading_time_minutes, truncated and, with includeLinks, a links list.
• Fetch details: final_url after redirects and the HTTP status.

Results land in the run's dataset, ready to export as JSON, CSV or Excel or to pass to the next Actor.

When should I use it?

• LLM and RAG pipelines: turn documentation, blog posts or a list of sources into Markdown chunks for embeddings or prompts.
• Link previews in bulk: enrich a bookmark list, a newsletter, a CMS or a chat export with titles, images and favicons.
• Content audits and SEO: check Open Graph and Twitter card tags, canonical URLs and missing images across many pages.
• After a crawler or search Actor: pass its URLs here to get clean text and metadata.

If you need one page at a time from your own code, use the Web Page API instead. It runs the same code behind a REST endpoint.

Input

{
  "urls": ["https://example.com/", "https://en.wikipedia.org/wiki/Markdown", "https://www.python.org/"],
  "mode": "both",
  "includeLinks": false,
  "includeImages": false,
  "maxChars": 100000
}

Field	Default	Description
urls	required	Public http(s) URLs (ports 80 and 443). Up to 1,000 per run; duplicates are removed
mode	both	both, preview or markdown
includeLinks	false	Keep hyperlinks in the Markdown and add a links list
includeImages	false	Keep images in the Markdown as 
maxChars	100000	Truncate each page's Markdown to this many characters (100 to 1,000,000)

Output

One row per page. This is the real row for https://www.python.org/ (abridged; checked 2026-10-24):

{
  "index": 2,
  "url": "https://www.python.org/",
  "mode": "both",
  "final_url": "https://www.python.org/",
  "status": 200,
  "title": "Welcome to Python.org",
  "description": "The official home of the Python Programming Language",
  "image": "https://www.python.org/static/opengraph-icon-200x200.png",
  "favicon": "https://www.python.org/static/favicon.ico",
  "lang": "en",
  "preview": { "site_name": "…", "canonical": "…", "images": ["…"], "favicons": ["…"], "twitter": "…", "fetch_ms": "…" },
  "markdown": "# Welcome to Python.org

## Get Started

Whether you're new to programming or an experienced developer, it's easy to learn …",
  "word_count": 269,
  "reading_time_minutes": 2,
  "truncated": false,
  "links": null,
  "errors": null
}

In both mode, if one part fails (for example robots.txt disallows the Markdown) the row still has the other part, and errors says what failed. URLs where nothing worked are not in the dataset: they are listed with their error code in the key-value store record FAILED and are not charged.

Is it safe to give it any URL?

Yes. It uses the same fetching code as the Web Page API, which is built for untrusted URLs:

• Private, loopback, link-local, CGNAT, reserved and cloud-metadata addresses (such as 169.254.169.254) are refused in every IPv4 and IPv6 spelling, as are internal names such as localhost and *.internal.
• DNS is resolved once, checked, and the connection goes to that exact IP, so DNS rebinding does not work.
• Every redirect hop (at most 5) is checked again. Only ports 80 and 443 are allowed.
• One 8-second deadline covers each fetch, and pages over 3 MB are refused.

Does it respect robots.txt?

Markdown does: it honours robots.txt for the token WebPageAPI (else *), and a disallowed page gets the error robots_disallowed. Link previews do not, the same as Slack, Discord and iMessage link unfurlers, because a preview reads only public head metadata.

How much does it cost?

It's pay per result: $0.0015 per page ($1.50 per 1,000 pages), plus a little Apify platform usage. both mode costs the same as one mode. Invalid, blocked, unreachable and robots-disallowed URLs are not charged. If you set a maximum spend on the run, it stops when that limit is reached.

FAQ

How do I convert a list of web pages to Markdown for ChatGPT, Claude or a RAG pipeline? Paste the URLs into urls, set mode to markdown (or keep both), run the Actor and export the dataset. Each row's markdown field holds the main content only.

How do I get link previews (title, image, favicon) for many URLs at once? Run it with mode set to preview. Each row has title, description, image and favicon, and the full Open Graph and Twitter card data in preview.

Does it render JavaScript? No. There is no headless browser, which keeps it fast and cheap. Pages that build their content with JavaScript return what the server sends; that is usually enough for a preview, but single-page apps may give little Markdown. For those, use a browser-based crawler.

How does it compare with other URL-to-Markdown tools? Most Apify Actors and APIs do either Markdown or link previews; this one returns both in one row for one price. Firecrawl and Jina Reader focus on Markdown, Microlink on previews and screenshots.

Is my data stored? The results stay in your own Apify storage, under your account's retention settings. Pages are fetched and parsed inside the run; nothing is sent to another service.

Related

• Web Page API: the same link preview and Markdown extraction as a REST API, one URL per call.
• Free URL to Markdown converter: try the extractor on a single page in your browser, no account needed.
• keith-apis-mcp: an MCP server that lets Claude, Cursor and other AI assistants read web pages as Markdown.