# Standby RAG Microservice API (Web-to-Markdown + JSON-LD) (`subimpact/standby-rag-api`) Actor

Sub-second warm HTTP microservice & batch scraper: strips HTML bloat, converts web pages into token-optimized Markdown, extracts Schema.org JSON-LD entities and token budgets for LLM/RAG pipelines.

- **URL**: https://apify.com/subimpact/standby-rag-api.md
- **Developed by:** [subimpact network](https://apify.com/subimpact) (community)
- **Features:** Limited permissions, Single-tenant, Server interface
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 clean markdown page converteds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Standby RAG Microservice API (Web-to-Markdown + JSON-LD)

> **Sub-second warm HTTP microservice & batch crawler:** strips HTML bloat, converts web pages into token-optimized Markdown, extracts Schema.org JSON-LD entities, and calculates token budgets for LLMs, AI agents, and RAG pipelines.

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-orange.svg)](https://apify.com)
[![Pricing](https://img.shields.io/badge/Pricing-Pay--Per--Event%20\($0.001%20/%20page\)-blue.svg)](#pricing)
[![Standby Mode](https://img.shields.io/badge/Standby%20Mode-Sub--Second%20HTTP-green.svg)](#standby-mode-sub-second-api)
[![TypeScript](https://img.shields.io/badge/TypeScript-Strict-blue.svg)](https://www.typescriptlang.org/)

***

### ⚡ Why Standby RAG API?

| Feature | Standard Web Scrapers | Firecrawl / Tavily | **Standby RAG API** |
| :--- | :--- | :--- | :--- |
| **Pricing** | High monthly minimums ($49+) | $2.00 – $5.00 / 1k pages | **$1.00 / 1k pages ($0.001 / page)** |
| **Execution Latency** | 15–45s (cold container boot) | 1–3s | **< 800ms (Warm Standby HTTP)** |
| **Direct HTML Conversion** | Not supported | Extra charges | **< 10ms instant conversion** |
| **Schema.org / JSON-LD** | Rarely parsed | Basic | **Full Graph entity extraction** |
| **Token Optimization** | Returns noisy raw HTML | Markdown | **Clean ATX Markdown + Token budget** |
| **Apify Standby Support** | ❌ No | ❌ No | **✅ Yes (`standbyUrl`)** |

***

### 🚀 Standby Mode (Sub-Second API)

When started with Standby enabled, this Actor runs continuously as an ultra-fast HTTP microservice behind your permanent Apify Standby URL (`https://<actor-name>.apify.actor/`).

#### Endpoint: `POST /convert`

Send either a live `url` to fetch or direct `html` to convert:

```bash
curl -X POST "https://your-actor.apify.actor/convert" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/article",
    "includeJsonLd": true
  }'
```

Or pass direct HTML for instant (<10ms) parsing:

```bash
curl -X POST "https://your-actor.apify.actor/convert" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "html": "<html><body><article><h1>Title</h1><p>Text</p></article></body></html>"
  }'
```

#### Response Example:

```json
{
  "success": true,
  "executionTimeMs": 34,
  "chargedUsd": 0.001,
  "data": {
    "url": "https://example.com/article",
    "title": "State of AI 2026",
    "description": "Comprehensive report on autonomous coding agents.",
    "author": "Antigravity Research",
    "publishedTime": "2026-10-01T00:00:00Z",
    "markdown": "# State of AI 2026\n\nComprehensive report on autonomous coding agents.\n\n## Highlights\n\n- Token-efficient retrieval\n- Schema.org entity linking",
    "jsonLd": [
      {
        "@context": "https://schema.org",
        "@type": "Article",
        "headline": "State of AI 2026"
      }
    ],
    "tokenEstimate": 420,
    "charCount": 1580,
    "wordCount": 245,
    "crawledAt": "2026-10-08T03:00:00.000Z"
  }
}
```

***

### 📦 Batch Mode (URL List Converter)

If started via the Apify Console or standard API with `startUrls`, the Actor will fetch exactly the URLs you provide (no link discovery — it does **not** follow links on a page), convert them to token-optimized Markdown, and store them in the default Apify Dataset. To process many pages, list all URLs in `startUrls` (up to `maxPages`, capped at 50,000):

```json
{
  "startUrls": [
    { "url": "https://news.ycombinator.com" },
    { "url": "https://techcrunch.com" }
  ],
  "maxPages": 25,
  "removeSelectors": [".newsletter-banner", "#cookie-consent"]
}
```

If every requested URL fails or no page yields usable content, the run fails with a clear status message; blank/bot-challenge pages are reported as uncharged error rows and are never billed.

***

### 💰 Pay-Per-Event (PPE) Pricing

- **Event:** `page-converted`
- **Price:** **$0.001 USD per page** ($1.00 per 1,000 pages), plus $0.00005 per run
- Zero monthly subscription commitments.
- Charged strictly per successful page conversion.
- Platform usage is included in the event price, not billed to you separately.

### How this compares

The actor runs as a warm standby HTTP service, so a single page conversion answers in well under a second instead of waiting for a cold container, and the same endpoint also returns Schema.org JSON-LD entities and a token budget alongside the Markdown. One batch mode covers URL lists. At $1.00 per 1,000 pages it is below the alternatives in the RAG browser lane: apify/rag-web-browser lists $2.28 per 1,000 fetches and $3.48 per 1,000 searches on the Free plan, s-r/rag-web-browser and datapilot/rag-web-browser-scraper $2.00, and parseforge/rag-web-browser $12.00.

| Actor | Price per 1,000 pages | Platform usage |
|---|---|---|
| subimpact/standby-rag-api | $1.00 | not billed separately |
| apify/rag-web-browser | $2.28 (fetch), $3.48 (search) | see listing |
| s-r/rag-web-browser | $2.00 | see listing |
| datapilot/rag-web-browser-scraper | $2.00 | see listing |
| parseforge/rag-web-browser | $12.00 | see listing |

Prices are Apify Store list prices on the Free plan, checked 24 October 2026. Competitor prices can change; check their listings.

***

### 🛠 Integration Examples

#### Python (LangChain / LlamaIndex / OpenAI)

```python
import requests

APIFY_TOKEN = "your_token_here"
## Public Store standby URL (creator-prefixed). For a DEDICATED (private)
## deployment use https://<actor-or-task-id>.apify.actor/convert instead.
STANDBY_URL = "https://subimpact--standby-rag-api.apify.actor/convert"

response = requests.post(
    STANDBY_URL,
    headers={"Authorization": f"Bearer {APIFY_TOKEN}"},
    json={"url": "https://example.com/blog/deep-dive"}
)

doc = response.json()["data"]
markdown_text = doc["markdown"]
token_budget = doc["tokenEstimate"]

print(f"Tokens: {token_budget}\nContent:\n{markdown_text[:200]}...")
```

#### TypeScript / Node.js

```typescript
import fetch from 'node-fetch';

// Public Store standby URL (creator-prefixed). For a DEDICATED (private)
// deployment use https://<actor-or-task-id>.apify.actor/convert instead.
const STANDBY_URL = 'https://subimpact--standby-rag-api.apify.actor/convert';

const res = await fetch(STANDBY_URL, {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_APIFY_TOKEN',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://news.ycombinator.com',
  }),
});

const { data } = await res.json();
console.log(`Title: ${data.title} (${data.tokenEstimate} tokens)`);
```

***

### 📄 License

Apache-2.0

### Run monitoring and privacy

If a run fails or times out, this Actor tells its developer the run ID and status, so we can fix problems fast. It never sends your input, your results or your account data.

# Actor input Schema

## `mode` (type: `string`):

Choose 'auto' (detects Standby HTTP server when warm, batch otherwise), 'standby' (HTTP server for sub-second REST requests), or 'batch' (crawls startUrls).

## `startUrls` (type: `array`):

Web page URLs to fetch, convert to token-optimized Markdown, and extract Schema.org entities in batch mode.

## `maxPages` (type: `integer`):

Maximum number of pages to process during a batch run.

## `removeSelectors` (type: `array`):

Custom CSS selectors to strip from HTML before Markdown conversion (e.g. '.newsletter-popup', '#cookie-banner').

## `includeJsonLd` (type: `boolean`):

Extract structured Schema.org JSON-LD entities (Articles, Products, FAQs, Organizations) for semantic graph enrichment.

## `includeHtml` (type: `boolean`):

Include the stripped HTML alongside clean Markdown in the dataset output (increases storage usage).

## `proxyConfiguration` (type: `object`):

Apify Proxy settings for accessing geo-blocked or protected websites.

## Actor input object example

```json
{
  "mode": "auto",
  "startUrls": [
    {
      "url": "https://news.ycombinator.com"
    }
  ],
  "maxPages": 10,
  "removeSelectors": [],
  "includeJsonLd": true,
  "includeHtml": false,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://news.ycombinator.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("subimpact/standby-rag-api").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://news.ycombinator.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("subimpact/standby-rag-api").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://news.ycombinator.com"
    }
  ]
}' |
apify call subimpact/standby-rag-api --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,subimpact/standby-rag-api"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/5C1bS2BdKe7Emii3l/builds/1L6UT47IlmHhvV48I/openapi.json
