Pricing

from $0.36 / 1,000 page extracteds

Try for free

Go to Apify Store

Website to Markdown — Content Extractor for LLMs & RAG

Try for free

Turn any website into clean Markdown and plain text for RAG pipelines, vector stores and LLM agents. Reads the sitemap or crawls links, strips navigation, cookie bars and footers, and returns one row per page with metadata, headings, links and optional chunks. Static HTML, no browser, no API key.

Pricing

from $0.36 / 1,000 page extracteds

Rating

0.0

(0)

Developer

Insight Solutions

Actor stats

Bookmarked

Total users

Monthly active users

9 days ago

Last modified

Categories

Developer tools

Marketing

Optional

True on a page or site row. False on a diagnostic row, which explains why one URL could not be read. Diagnostic rows are never charged.

Type:boolean | null

Example:

true

Row type

rowType

Optional

page one extracted page. site the per-website summary, written after the pages it summarises. diagnostic a free row explaining a failure.

Type:string | null

Example:

page

Input entry

input

Optional

The startUrls entry this row came from, exactly as you supplied it.

Type:string | null

Example:

https://docs.apify.com/platform/actors

Website

website

Optional

The registrable domain this row belongs to, lower-cased. This is the column to group a multi-site run on.

Type:string | null

Example:

apify.com

URL

url

Optional

On a page row, the URL that was requested. On a site row, the origin the website answered on. On a diagnostic row, the URL that could not be read.

Type:string | null

Example:

https://docs.apify.com/platform/actors

Final URL

finalUrl

Optional

The URL the response actually came from. Differs from url when the page redirected; rows are deduplicated on this, so a redirect that lands two queued URLs on one page produces one row and one charge.

Type:string | null

Example:

https://docs.apify.com/platform/actors

HTTP status

statusCode

Optional

The status the page answered with. Only 2xx pages become page rows; a 4xx or 5xx is an error page rather than content and gets a free http-error diagnostic instead.

Type:integer | null

Example:

200

Content type

contentType

Optional

The raw Content-Type response header.

Type:string | null

Example:

text/html; charset=utf-8

Response time (ms)

responseTimeMs

Optional

Wall-clock milliseconds for the request that produced this body, measured at the transport. Includes proxy time, because a reader's request would too.

Type:integer | null

Example:

817

Title

title

Optional

og:title if the page publishes one, else the <title> element, else the first heading in the extracted content.

Type:string | null

Example:

Actors | Platform | Apify Documentation

Description

description

Optional

The meta description, or og:description when there is no meta description.

Type:string | null

Example:

Learn how to develop, run and share serverless cloud programs.

Site name

siteName

Optional

og:site_name, when the page publishes one.

Type:string | null

Example:

Apify Documentation

Author

author

Optional

From the author meta tag, article:author, a rel=author link, or the author.name of a JSON-LD block — whichever the page publishes first, in that order.

Type:string | null

Example:

Jo Smith

Published at

publishedAt

Optional

article:published_time, JSON-LD datePublished, or the first <time datetime> in the content. Reported exactly as the page spells it, which is usually but not always ISO 8601.

Type:string | null

Example:

2026-02-03T09:00:00Z

Modified at

modifiedAt

Optional

article:modified_time or JSON-LD dateModified, as the page spells it.

Type:string | null

Example:

2026-03-04T10:30:00Z

Language

language

Optional

The lang attribute of the <html> element, lower-cased.

Type:string | null

Example:

Canonical URL

canonical

Optional

The rel=canonical link the page declares, resolved to an absolute URL.

Type:string | null

Example:

https://docs.apify.com/platform/actors

Lead image

image

Optional

og:image, resolved to an absolute URL. The page's own choice of social preview, not the first image in the body.

Type:string | null

Example:

https://docs.apify.com/img/og.png

Keywords

keywords

Optional

The keywords meta tag, split on commas and trimmed. Empty on the majority of pages, which no longer publish one.

Type:(string | null)[] | null

Example:

markdown,tables,code

JSON-LD types

jsonLdTypes

Optional

Every @type found in the page's JSON-LD blocks, deduplicated, schema.org/ prefix removed. Article, BlogPosting, Product and FAQPage are the useful ones for deciding what a page is.

Type:(string | null)[] | null

Example:

Article,Person

Markdown

markdown

Optional

The page's content as Markdown: headings, paragraphs, nested lists, fenced code with a language where the page declares one, GitHub pipe tables, blockquotes, links and images. Navigation, headers, footers, sidebars, cookie bars, share widgets and scripts are gone. Null when What to return is set to plain text only.

Type:string | null

Example:

# Actors Actors are serverless cloud programs...

Plain text

text

Optional

The same content with the Markdown syntax stripped: headings on their own line, paragraphs separated by blank lines, list items prefixed with - , code blocks unfenced. Null when What to return is set to Markdown only.

Type:string | null

Example:

Actors Actors are serverless cloud programs...

Extracted HTML

html

Optional

The markup of the content region the Actor chose, with boilerplate already removed. Only filled when Also return the extracted HTML is on.

Type:string | null

Example:

<article><h1>Actors</h1>...</article>

Words

wordCount

Optional

Words of extracted content — tokens carrying at least one letter or digit. A page with fewer than 20 is reported as an empty diagnostic rather than a row, because it is almost always a JavaScript-rendered shell.

Type:integer | null

Example:

295

Characters

charCount

Optional

Characters of extracted plain text, Markdown syntax excluded.

Type:integer | null

Example:

1865

Token estimate

tokenEstimate

Optional

Characters divided by four, rounded up. A rule of thumb for budgeting an embedding run, not a tokenizer: your model's own count will differ, usually by less than 15% on English prose and by more on code.

Type:integer | null

Example:

520

Reading time (minutes)

readingTimeMin

Optional

Words divided by 230, rounded, minimum 1.

Type:integer | null

Example:

Headings

headings

Optional

Every heading inside the extracted content, in document order, as {level, text} with level 1–6. The outline of the page, without the navigation headings around it.

Type:(object | null)[] | null

Example:

[object Object],[object Object]

Links

links

Optional

Up to 500 links found inside the extracted content, as {text, url, internal} with the URL made absolute. internal is true when the link stays on the same registrable domain. Links in the navigation and footer are not here — they were removed with the boilerplate — but they are still followed by the crawl.

Type:(object | null)[] | null

Example:

[object Object]

Internal links

internalLinkCount

Optional

How many of the content links stay on the same registrable domain.

Type:integer | null

Example:

External links

externalLinkCount

Optional

How many of the content links point somewhere else.

Type:integer | null

Example:

Images

images

Optional

Up to 100 images from the extracted content as {src, alt}, with src made absolute. Spacers, 1×1 tracking beacons and data: URIs are excluded.

Type:(object | null)[] | null

Example:

[object Object]

Chunks

chunks

Optional

Only when Chunk size is set. Each entry is {index, text, headingPath, charCount, tokenEstimate}, split at block boundaries, never through a fenced code block or a table, with headingPath carrying the H1 > H2 > H3 trail the chunk sits under. Ready to embed as-is.

Type:(object | null)[] | null

Example:

[object Object]

Chunks

chunkCount

Optional

How many chunks the page produced. Null when chunking is off.

Type:integer | null

Example:

Extraction method

extractionMethod

Optional

Which strategy found the content. article the page's <article> element. main its <main> or [role=main]. selector a well-known content id or class such as .markdown-body or .entry-content. density a text-density score, for pages with none of the above. body the last resort, the whole body with boilerplate stripped — worth a look when a row reads oddly.

Type:string | null

Example:

article

Boilerplate ratio

boilerplateRatio

Optional

The share of the page's visible text that was not kept: 0.30 means 30% of what the server sent was navigation, footers and widgets. High on a thin page with a big site template, near zero on a long article.

Type:number | null

Example:

0.294

Crawl depth

depth

Optional

0 for a URL you listed or one that came out of a sitemap, 1 for a link found on a depth-0 page, and so on.

Type:integer | null

Example:

Discovered via

discoveredVia

Optional

input you listed this URL, sitemap it came out of the site's sitemap, link it was found on another page.

Type:string | null

Example:

sitemap

Sitemap lastmod

sitemapLastmod

Optional

The <lastmod> the sitemap published for this URL, when it came from one. Useful for re-crawling only what changed.

Type:string | null

Example:

2026-02-03T11:00:00+00:00

Pages extracted

pagesExtracted

Optional

site rows only: how many pages of this website produced a row.

Type:integer | null

Example:

How pages were found

discovery

Optional

site rows only: sitemap, link (crawled from the home page because there was no sitemap) or input.

Type:string | null

Example:

sitemap

Sitemap URLs

sitemapUrlCount

Optional

site rows only: how many URLs were collected from the site's sitemaps before the page budget cut collection short.

Type:integer | null

Example:

Blocked by robots.txt

robotsBlocked

Optional

site rows only: how many discovered URLs were skipped because robots.txt disallows them. Never fetched, never billed.

Type:integer | null

Example:

Total words

totalWords

Optional

site rows only: the sum of wordCount across this website's pages.

Type:integer | null

Example:

18450

Total token estimate

totalTokens

Optional

site rows only: the sum of tokenEstimate, for budgeting an embedding run over the whole crawl.

Type:integer | null

Example:

26400

Average boilerplate ratio

avgBoilerplateRatio

Optional

site rows only: the mean boilerplateRatio across this website's pages. A number near 1 means the site is mostly template, which is usually a sign the pages are thin rather than that extraction went wrong.

Type:number | null

Example:

0.31

Failures

failures

Optional

site rows only: how many URLs of this website could not be turned into a row. Each has its own free diagnostic row.

Type:integer | null

Example:

What happened

error

Optional

Diagnostic rows only: a plain-language sentence explaining why this URL produced no content.

Type:string | null

Example:

That URL answered with application/pdf, not a web page.

Error type

errorType

Optional

unreachable, blocked, timeout, invalid-input, not-html, http-error, too-large or empty. Filter the dataset on this to see what a crawl missed and why.

Type:string | null

Example:

not-html

Read at

scrapedAt

Optional

ISO 8601 timestamp of when this row was produced.

Type:string | null

Example:

2026-09-11T21:30:00.000Z

Source

source

Optional

Where this row came from: input, sitemap, link or site.

Type:string | null

Example:

sitemap

Source URL

sourceUrl

Optional

The exact URL the content was read from.

Type:string | null

Example:

https://docs.apify.com/platform/actors

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

338

5.0

(2)

Website Content Crawler — Text, Markdown & HTML for AI/LLM

hichemdev/website-content-crawler

Crawl any website and extract clean text, Markdown, and HTML from every page — ready for LLM, RAG, and AI ingestion.

Hichem Ben Moussa

Website Content Crawler

jasondev/website-content-crawler

A powerful web crawler that extracts text content from websites, optimized for AI models, Large Language Models (LLMs), vector databases, and Retrieval-Augmented Generation (RAG) pipelines.

Jason Giang

Website Content Extractor for RAG: Markdown, HTML, Text

nezha/website-content-crawler

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.

nezha

Website Content Crawler — Text, Titles & Metadata

hipersoft/website-content-crawler

Extract clean, readable content from any list of websites: page title, meta description, headings, main body text, word count and link/image counts. Optional same-domain crawl. Bulk-ready. Great for LLM/RAG ingestion, content audits and research.

hiper soft

Website Content Crawler — Markdown for LLM & RAG

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Yuliia Kulakova

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Locomint

Website Content Extractor

tropical_prune/website-content-extractor

CAPABILITIES: extract_content, convert_to_markdown, batch_urls, extract_metadata. INPUT: URLs (single or array), with optional selectors and output format. OUTPUT: structured JSON with title, text, metadata, word_count. FORMATS: json, markdown, text. PRICING: PPE $0.001/page.

Bado

Knowledge Intelligence Engine — Website to Markdown for RAG

ryanclinton/website-content-to-markdown

Turn any website, documentation site or help centre into a retrieval-ready knowledge corpus for RAG and AI search. Clean Markdown plus chunks, change detection, deduplication, retrieval scoring, version awareness and a full corpus audit, in one run.

Ryan Clinton

AI Website Content Markdown Scraper

quaking_pail/ai-website-content-markdown-scraper

This Apify Actor, "Website Content Crawler with Markdown Extraction," is designed to perform a comprehensive crawl of specified websites, extract their text content, convert it into Markdown format, and store it in a structured dataset. The extracted content is suitable for feeding LLMs.