Webpage To Markdown Converter

What it does

Converts web pages into clean Markdown format. For each provided URL, the actor fetches the HTML content, extracts the main body, and converts HTML elements to their Markdown equivalents including headers, links, lists, bold and italic text, images, and paragraphs. Outputs the page title, converted Markdown content, word count, and fetch timestamp. Useful for content archival, documentation generation, knowledge base building, and converting web content for use in Markdown-based systems like GitHub or static site generators.

This Apify actor automates the collection of data from a public API or website, extracting structured information and saving it directly into an Apify dataset. It handles pagination automatically where applicable, supports configurable result limits, and includes robust error handling with timeouts on all HTTP requests. The actor is designed for reliability: it validates inputs, applies sensible defaults, and produces a fallback record when no results are found, so your downstream workflows never receive an empty dataset. Built on the Apify SDK with native Node.js 20 fetch for lightweight, fast execution without browser overhead.

Why use it

Manually collecting data from web APIs and websites is tedious and error-prone. This actor eliminates that burden by running in the cloud on the Apify platform, where it can be scheduled, integrated with webhooks, or chained with other actors. Whether you are conducting research, building a knowledge base, monitoring data sources, or feeding data into an analytics pipeline, this actor gives you structured, ready-to-use JSON output with zero browser overhead. It uses lightweight HTTP requests instead of a full browser, which makes it fast and cost-effective. Every request includes a 120-second timeout to prevent hanging, and all string fields are null-checked for data consistency.

Input parameters

• urls (array, required): List of webpage URLs to convert to Markdown. Default: ["https://en.wikipedia.org/wiki/Web_scraping"].
• maxContentLength (integer, optional): Maximum number of characters of Markdown content per page. Default: 50000. Range: 1000-200000.

All inputs are validated at startup with sensible defaults applied when values are missing. The actor will log warnings for any misconfigured options and continue with safe defaults rather than failing outright.

Output data

Each item in the output dataset contains the following fields:

• url: The URL of the converted page
• title: Page title from the title tag
• markdown: The converted Markdown content (truncated to maxContentLength)
• wordCount: Number of words in the Markdown output
• fetchedAt: ISO timestamp when the page was fetched

All string fields are null-checked; missing values are stored as null rather than undefined.

Example output

{
    "url": "https://en.wikipedia.org/wiki/Web_scraping",
    "title": "Web scraping - Wikipedia",
    "markdown": "# Web scraping\n\nWeb scraping is the process of...",
    "wordCount": 3500,
    "fetchedAt": "2024-01-15T12:00:00.000Z"
}

Pricing

This actor is priced on a usage basis:

• $0.01 per result returned in the dataset.
• $0.005 per actor start (fixed platform fee).

For example, scraping 500 results would cost approximately $5.005. Apify provides free monthly credits for new users, so you can try the actor at no charge. Actual costs depend on the number of results, API response times, and memory allocation. You can control costs by setting the maxResults parameter to limit the number of results collected per run. For high-volume use cases, consider running the actor on a schedule during off-peak hours to optimize platform resource usage.

More scrapers from brave_paradise

Check out other useful scrapers built by brave_paradise:

• OpenAQ Air Pollution Data Scraper
• Open-Meteo Historical Weather Scraper
• GeoNames Place Search Scraper

Visit the brave_paradise profile on Apify to see the full catalogue of actors.