# URL to Markdown + Metadata for AI & RAG 💰$2/1k (`flyingnimbus/url-to-markdown-metadata`) Actor

Turn any web page into clean Markdown plus structured metadata for LLMs and RAG pipelines. You get the main content, heading outline, OpenGraph and Twitter tags, JSON-LD, canonical URL, language, word count and an approximate token count. It uses plain HTTP, so it is fast and cheap.

- **URL**: https://apify.com/flyingnimbus/url-to-markdown-metadata.md
- **Developed by:** [Rakshit Solanki](https://apify.com/flyingnimbus) (community)
- **Categories:** AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## URL to Markdown + Metadata

Turn any web page into clean Markdown plus structured metadata for LLMs and RAG pipelines. You get the main content, heading outline, OpenGraph and Twitter tags, JSON-LD, canonical URL, language, word count and an approximate token count. It uses plain HTTP, so it is fast and cheap.

### What does URL to Markdown + Metadata do?

- Main-content extraction that drops navigation and footers
- Absolute links and image URLs
- OpenGraph, JSON-LD, author, dates and canonical URL
- Heading outline and optional link list
- Token estimate to size your prompts

### How to use it

1. Click **Try for free** and open the **Input** tab.
2. Fill in the fields below, or keep the defaults for a quick test.
3. Press **Start**. Results appear in the **Output** tab within seconds to a few minutes.
4. Download as JSON, CSV or Excel, or call the actor from the API, a schedule, Make, Zapier or an AI agent through the Apify MCP server.

### Input

| Field | Type | Description |
|---|---|---|
| `urls` | array | Web pages to convert, one per line. |
| `mainContentOnly` | boolean | Drop navigation, headers, footers and sidebars. |
| `includeLinks` | boolean | Return all links with internal/external flag. |
| `maxCharsPerPage` | integer | Truncate long pages (0 = no limit). |

#### Example input

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "mainContentOnly": true,
  "includeLinks": false,
  "maxCharsPerPage": 100000
}
```

### Output

Each result is a flat JSON object that you can open in the **Output** tab, export to CSV or Excel, or read through the API. See the dataset views for the main columns.

### How much does it cost to use URL to Markdown + Metadata?

Pricing is pay per event: **$0.002 per page**. You pay only for results saved to the dataset, plus nothing extra for platform usage on the free plan limits. Apify's free monthly credit lets you test it before paying.

### FAQ

**Does it run JavaScript?**  No. Pages that render content only in the browser return little text and are flagged likelyJsRendered.

**Is it allowed to scrape any site?**  You are responsible for respecting each site's terms and robots.txt.

**How is it different from a crawler?**  It converts the URLs you list; it does not follow links.

### Is it legal?

The actor reads public, open-licence or government data only and does not collect private information. You are responsible for using the results in line with the source's terms and applicable laws.

### Use it with AI agents (MCP)

This actor is agent-ready: it is a plain tool with a documented JSON input and a flat JSON output, so Claude, ChatGPT, Cursor and other agents can call it through the [Apify MCP server](https://mcp.apify.com).

Add it to an MCP client with this URL (Apify account or agentic payment required):

```
https://mcp.apify.com/?tools=flyingnimbus/url-to-markdown-metadata
```

Example prompt: "Use the URL to Markdown + Metadata tool with the example input below and summarise the results."

Calls are pay per event ($0.002 per page), so an agent can estimate the cost before it runs. Empty or failed lookups return a status record instead of failing the run.

### More actors by flyingnimbus

- [Property Details by Address API](https://apify.com/flyingnimbus/property-details-by-address-api)
- [Schools Near Address Finder API](https://apify.com/flyingnimbus/schools-near-address-finder)
- [School Rating Converter & Equivalence API](https://apify.com/flyingnimbus/school-rating-normalizer)
- [Building Permits Multi-City API](https://apify.com/flyingnimbus/building-permits-multi-city)
- [Company Registry Lookup (LEI + SEC)](https://apify.com/flyingnimbus/company-registry-lookup)
- [Restaurant Inspections API - NYC, Chicago, Toronto](https://apify.com/flyingnimbus/restaurant-inspections-multi-city)
- [CVE Risk Enricher - NVD, CISA KEV & EPSS](https://apify.com/flyingnimbus/cve-risk-enricher)
- [SEC Insider Trades (Form 4) - Buys & Sells](https://apify.com/flyingnimbus/sec-insider-trades)
- [SEC 13F Holdings Tracker](https://apify.com/flyingnimbus/sec-13f-holdings)
- [EU Company Registry Search](https://apify.com/flyingnimbus/eu-company-registry-search)
- [Canada Recalls & Safety Alerts Monitor](https://apify.com/flyingnimbus/canada-recalls-monitor)
- [Sitemap & Robots.txt Analyzer](https://apify.com/flyingnimbus/sitemap-robots-analyzer)
- [Email Validator - MX, SPF & DMARC](https://apify.com/flyingnimbus/email-mx-validator)
- [Website Tech Stack Detector](https://apify.com/flyingnimbus/tech-stack-detector)
- [PDF Text & Table Extractor](https://apify.com/flyingnimbus/pdf-table-extractor)
- [SSL Certificate Expiry Checker](https://apify.com/flyingnimbus/ssl-certificate-checker)
- [VIN Decoder + Recalls API](https://apify.com/flyingnimbus/vin-decoder-recalls)
- [ClinicalTrials.gov Search API](https://apify.com/flyingnimbus/clinical-trials-search)
- [US Federal Contracts & Grants (USAspending)](https://apify.com/flyingnimbus/federal-contract-awards)
- [Federal Register Search API](https://apify.com/flyingnimbus/federal-register-search)
- [FDA Recalls & Drug Labels API](https://apify.com/flyingnimbus/openfda-recalls-labels)
- [Eurostat Data API](https://apify.com/flyingnimbus/eurostat-data-api)
- [UK Government Tenders (Contracts Finder)](https://apify.com/flyingnimbus/uk-tenders-search)

### Feedback

Found a bug or need another field or city? Open an issue on the actor's **Issues** tab and it will be looked at quickly.

# Actor input Schema

## `urls` (type: `array`):

Web pages to convert, one per line.

## `mainContentOnly` (type: `boolean`):

Drop navigation, headers, footers and sidebars.

## `includeLinks` (type: `boolean`):

Return all links with internal/external flag.

## `maxCharsPerPage` (type: `integer`):

Truncate long pages (0 = no limit).

## Actor input object example

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "mainContentOnly": true,
  "includeLinks": false,
  "maxCharsPerPage": 100000
}
```

# Actor output Schema

## `results` (type: `string`):

One row per result.

## `overview` (type: `string`):

Key columns.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://en.wikipedia.org/wiki/Web_scraping"
    ],
    "mainContentOnly": true,
    "includeLinks": false,
    "maxCharsPerPage": 100000
};

// Run the Actor and wait for it to finish
const run = await client.actor("flyingnimbus/url-to-markdown-metadata").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://en.wikipedia.org/wiki/Web_scraping"],
    "mainContentOnly": True,
    "includeLinks": False,
    "maxCharsPerPage": 100000,
}

# Run the Actor and wait for it to finish
run = client.actor("flyingnimbus/url-to-markdown-metadata").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "mainContentOnly": true,
  "includeLinks": false,
  "maxCharsPerPage": 100000
}' |
apify call flyingnimbus/url-to-markdown-metadata --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,flyingnimbus/url-to-markdown-metadata"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/A7zGJwQMmPKBXYePp/builds/rmk9ipk3AaAgIMOP7/openapi.json