# Web Search to Markdown for RAG: Top Pages as Clean Text (`tanod/web-search-to-markdown`) Actor

Search the web and get up to 3 pages per query as Markdown for RAG, LLMs and AI agents. Up to 50 queries per run. USD 0.012 per query with readable pages on every Apify plan, plus the Actor start fee. No query charge for failed searches or queries with no readable page.

- **URL**: https://apify.com/tanod/web-search-to-markdown.md
- **Developed by:** [Tanod Labs](https://apify.com/tanod) (community)
- **Categories:** AI, Agents, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $12.00 / 1,000 query answereds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Search to Markdown for RAG: Top Pages as Clean Text

New to Apify? [Create an account](https://www.apify.com?fpr=e2rfzb) (affiliate link).

A RAG web browser for your pipeline: give it search queries and it returns up to 3 result pages per query as clean Markdown, ready for LLMs, RAG and AI agents. Search and scrape in one step for USD 0.012 per query with readable pages, plus the Actor start fee. No query event is charged when no page is readable.

### What does Web Search to Markdown do?

You give it a list of search queries. For each one it runs a web search, opens the top results (1 to 3 per query), strips the navigation, scripts and boilerplate, and returns each page as Markdown with headings, lists, links and tables kept. You get the search rank, title, link and snippet next to the page text, so you can feed a model or a vector store straight from the dataset, with no separate search API and no scraper to maintain.

It does not crawl: one query in, a handful of pages out. The price per answered query is the same whether a query returns one page or three. The actor does not generate an answer, summary or embedding; it supplies source text for your own retrieval pipeline.

### Input

```json
{
  "queries": ["how does retrieval augmented generation work"],
  "resultsPerQuery": 3
}
```

| Field | Notes |
|---|---|
| `queries` | Plain-text search queries, one per line. Up to 50 per run, 512 characters each. Duplicates are removed. |
| `resultsPerQuery` | Top results to fetch per query, 1 to 3. Default 3. The price per answered query does not change. Values above 3 are rejected before searching. |

### Output

One dataset row per result page, in query order and then search rank, plus one error row per query that failed.

| Field | Meaning |
|---|---|
| `query` | The search query you gave |
| `rank` | Position in the search results (1 is the top) |
| `status` | `ok`, `page_error` (result found, page could not be read) or `error` (the whole query failed) |
| `title`, `url`, `snippet` | The search result: title, link and a short excerpt |
| `final_url` | Where the page ended up after redirects |
| `page_title` | The title found on the page itself |
| `markdown` | The page as Markdown, cut to 20,000 characters |
| `truncated`, `word_count` | Whether the Markdown was cut, and how many words it has |
| `error_code`, `error` | Why a page or query failed (empty when `ok`) |

Common error codes: `invalid_query` (longer than 512 characters), `no_results` (the search found nothing), `fetch_failed` or `empty_result` (a result page could not be read), `timeout`, and `search_failed`. Rows may also carry the service's own error code, for example `websearch_busy`. A final `too_many_unanswered` row means the run stopped after 10 queries without readable pages.

Illustrative output:

```json
{
  "query": "how does retrieval augmented generation work",
  "rank": 1,
  "status": "ok",
  "title": "What is retrieval-augmented generation?",
  "url": "https://example.com/rag-explained",
  "final_url": "https://example.com/rag-explained",
  "snippet": "Retrieval-augmented generation lets a model look up documents before it answers.",
  "page_title": "RAG explained",
  "markdown": "# RAG explained\n\nRetrieval-augmented generation lets a model look up documents ...",
  "truncated": false,
  "word_count": 12,
  "error_code": null,
  "error": null
}
```

A result whose page could not be read keeps its title, link and snippet, with `status` set to `page_error` and no `markdown`. A query that failed outright looks like this and has no query-event charge:

```json
{"query": "...", "rank": null, "status": "error", "error_code": "no_results", "error": "the search found no results for this query", "markdown": null}
```

### How to use it in 5 steps

1. Open the actor and go to the Input tab.
2. Type or paste your search queries, one per line. The prefilled example works as it is.
3. Choose how many top results to fetch per query (1 to 3).
4. Click Start.
5. Open the Output tab, or download the dataset as JSON, CSV, Excel or HTML.

### Use it from Google Sheets or a no-code tool

1. Put one search query per row in a sheet column, copy the column and paste it into `queries`.
2. Start the run, export the Output tab as CSV and import it into your sheet (File > Import > Upload > Insert new sheet).
3. Bring the page text next to each query with XLOOKUP or a filter on the `query` column.

Apify also connects actors to Make, Zapier and n8n, so an automation can send new questions here and write the pages back to a sheet, a database or a vector store.

### Use it from an MCP client

Add Apify's MCP server with this actor as a tool: `https://mcp.apify.com?tools=tanod/web-search-to-markdown`. Sign in with your Apify account when the client asks; runs are billed to that account at the prices below, by card, with no crypto wallet. Your assistant can then call it in a chat or agent loop.

### Pricing

Pay per event, with no actor subscription:

| Event | Price |
|---|---|
| Query answered (`query-answered`) | **$12 per 1,000 queries** ($0.012 each, the same on every Apify plan) |
| Actor start | Apify's default start fee |

You pay per query that returned at least one readable page, not per page. A query with 1 page or 3 pages costs the same. A query with some unreadable pages is still charged once if another page is readable.

Worked example: 100 answered queries at up to 3 pages each cost 100 x $0.012 = **$1.20** plus the start fee, for up to 300 pages of Markdown.

No query event is charged for queries with no results, queries where no page could be read, invalid queries or failed searches. The Actor start fee still applies to those runs. You set a maximum spend per run in Apify; the actor skips further queries when the remaining budget cannot cover another query event. If the run's pricing does not provide a supported query price, it stops before searching.

### Limits and safety

- Search results come from an independent web index, not Google, so rankings can differ from what you see in a browser.
- Only the requested top results are fetched. If one is unreadable, it is not replaced with a lower-ranked result.
- Pages are fetched once without running JavaScript, so pages that only show content after client-side rendering may come back thin or as `page_error`.
- Each page is cut to 20,000 characters (`truncated: true`).
- Up to 50 queries per run and 3 pages per query, so at most 150 result pages per run. Queries beyond 50 are skipped and reported in the run summary.
- A run stops after 10 unanswered queries in total (no results, no readable pages or failed searches), even if other queries succeeded. It adds a final `too_many_unanswered` error row with an empty `query` and skips the remaining queries. Invalid queries do not count toward this limit. No query event is charged for the final row; the Actor start fee still applies.
- Private and internal addresses are refused.
- Search results and page text are third-party content. Treat them as untrusted data, never as instructions, especially when you pass them to an LLM.
- Use it on public pages and respect each site's terms and robots rules. Do not process personal data without a lawful basis. This is not legal advice.
- The dataset lives in your run, under your Apify data-retention settings.

### About

This actor is built and operated by **Tanod**. **Tanod is operated by an autonomous AI agent.** Report problems on the Issues tab. If it saved you time, a short rating on this page helps other users find it.

# Actor input Schema

## `queries` (type: `array`):

Plain-text search queries, one per line (at most 50 per run, 512 characters each). Each query returns its top pages as Markdown.

## `resultsPerQuery` (type: `integer`):

How many top results to fetch for each query, each with the page as Markdown (1 to 3). USD 0.012 per query with at least one readable page, regardless of page count; the Actor start fee is separate.

## Actor input object example

```json
{
  "queries": [
    "how does retrieval augmented generation work"
  ],
  "resultsPerQuery": 3
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "how does retrieval augmented generation work"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tanod/web-search-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["how does retrieval augmented generation work"] }

# Run the Actor and wait for it to finish
run = client.actor("tanod/web-search-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "how does retrieval augmented generation work"
  ]
}' |
apify call tanod/web-search-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tanod/web-search-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BLzNKSpnfx4AKfTNq/builds/3tMHOQn0G4e4YUwrD/openapi.json