# AI & RAG Clean Markdown + JSON-LD + Tables Web Extractor (`super_vegetable/ai-rag-markdown-structured-extractor`) Actor

Converts any list of URLs into clean, token-efficient Markdown, extracts JSON-LD/Schema.org data, HTML tables as JSON, and page metadata. Built for AI Agents, RAG pipelines, and LLM knowledge bases.

- **URL**: https://apify.com/super_vegetable/ai-rag-markdown-structured-extractor.md
- **Developed by:** [Albert](https://apify.com/super_vegetable) (community)
- **Categories:** AI, Developer tools, Open source
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 extracted url / domain results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI & RAG Clean Markdown + JSON-LD + Tables Web Extractor

Convert any list of web URLs into **clean, token-efficient Markdown**, structured **HTML tables (as JSON)**, and embedded **Schema.org / JSON-LD metadata** in seconds.

Designed specifically for **AI Agents, RAG Pipelines (LangChain / LlamaIndex), n8n / Make automations, and LLM Knowledge Bases**.

***

### Why Use This Extractor Instead of Raw HTML Scraping?

1. **Saves 70%–85% on LLM Token Costs:** Automatically strips `<nav>`, `<footer>`, `<script>`, `<style>`, ads, and cookie banners before converting to Markdown.
2. **Preserves Structured Tables as JSON:** Financial tables, pricing comparisons, and technical specs are extracted into clean JSON arrays alongside the Markdown.
3. **Automatic Schema.org / JSON-LD Extraction:** Captures product prices, ratings, FAQs, and article metadata embedded in `<script type="application/ld+json">`.
4. **Built-in Token Estimator:** Every output record includes `estimated_tokens` and `word_count` so your RAG pipeline can chunk documents safely.

***

### Example Input

```json
{
  "urls": [
    "https://example.com/pricing",
    "https://example.com/documentation"
  ],
  "includeTables": true,
  "includeSchemaOrg": true,
  "maxCharsPerUrl": 50000
}
```

### Example Output

```json
{
  "url": "https://example.com/benchmark",
  "status_code": 200,
  "title": "Hetzner vs AWS Cost Report 2026",
  "description": "Benchmark of cloud vs self-hosted servers.",
  "word_count": 14,
  "estimated_tokens": 19,
  "tables_count": 1,
  "json_ld_count": 1,
  "markdown": "# Cloud vs Self-Hosting Benchmark\n\nMigrating PostgreSQL from **AWS RDS** to Docker saves up to 82%.",
  "tables": [
    [
      ["Provider", "Monthly Cost"],
      ["AWS RDS 16GB", "$296"],
      ["Hetzner Cloud 16GB", "$19"]
    ]
  ],
  "json_ld": [
    {
      "@context": "https://schema.org",
      "@type": "TechArticle",
      "headline": "Cloud Cost Benchmark"
    }
  ]
}
```

***

### How to Deploy to Apify Store in 2 Minutes (Monetization Guide)

1. Install Apify CLI once:
   ```bash
   npm install -g apify-cli
   apify login
   ```
2. Push this Actor from `/home/broquil/pasivo/actors/ai-rag-web-extractor`:
   ```bash
   cd /home/broquil/pasivo/actors/ai-rag-web-extractor
   apify push
   ```
3. In Apify Console -> **Publication**, enable **Monetization** and select **Rental ($19/month with 3-day free trial)** or **Pay per result ($1.50 per 1,000 results)**.

# Actor input Schema

## `urls` (type: `array`):

List of web page URLs to extract clean Markdown and structured data from.

## `includeTables` (type: `boolean`):

Parses all HTML <table> elements into clean structured JSON arrays.

## `includeSchemaOrg` (type: `boolean`):

Extracts embedded JSON-LD structured data (products, prices, FAQs, articles).

## `maxCharsPerUrl` (type: `integer`):

Truncates output cleanly to control LLM token budgets.

## `timeoutSeconds` (type: `integer`):

HTTP timeout per page.

## Actor input object example

```json
{
  "urls": [
    "https://news.ycombinator.com",
    "https://docs.python.org/3/whatsnew/3.12.html"
  ],
  "includeTables": true,
  "includeSchemaOrg": true,
  "maxCharsPerUrl": 50000,
  "timeoutSeconds": 20
}
```

# Actor output Schema

## `overview` (type: `string`):

Main dataset summary view of extracted Markdown and structured metadata

## `rawJson` (type: `string`):

Direct JSON endpoint containing markdown, tables, and json_ld arrays

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("super_vegetable/ai-rag-markdown-structured-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("super_vegetable/ai-rag-markdown-structured-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call super_vegetable/ai-rag-markdown-structured-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,super_vegetable/ai-rag-markdown-structured-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KQndYSniuwpGt7GCn/builds/vdsaPVPYL6hSIq81v/openapi.json