# RAG Web Search: Search Results as Markdown for LLMs (`torix33/rag-web-search`) Actor

Web search for AI agents, LLMs and RAG: run any query and get the top result pages as clean Markdown with title, snippet, author, dates and word count. Fast HTTP, no browser, ideal for tool calls and MCP. Pay per result.

- **URL**: https://apify.com/torix33/rag-web-search.md
- **Developed by:** [Torix](https://apify.com/torix33) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.10 / 1,000 result scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RAG Web Search — Search Results as Markdown for LLMs & AI Agents

Give it a query, get back the **top web results as clean Markdown** — ready to drop into an LLM prompt, a RAG pipeline or an AI agent's tool call.

Every result contains the **search rank, title, URL, snippet** and (with page fetching on) the page's **main content as Markdown** (Readability: no menus, ads or cookie banners) plus **description, author, published/modified dates, language and word count**.

### How to run it in 3 steps

1. **Add your input.** In **Search queries** paste search questions, one per line — e.g. `best vector database for RAG`. The form is already filled with a working example, so you can skip this step for a first try.
2. **Press Start.** The prefilled run returns 5 results in about 5 s and costs about **$0.015**. Set **Max results (cost cap)** to cap the cost of bigger runs; you only pay for results actually delivered.
3. **Get the data.** Open the **Output** tab and export JSON, CSV or Excel — or run it from the API, Make, Zapier, n8n or an AI agent (see below).

### Features

- **Fast:** pure HTTP — 4 queries × 8 pages in ~6 seconds. No browser to boot.
- **Always N results:** pages that fail, block or are too thin are replaced by the next search result (and never charged).
- **LLM-friendly controls:** `maxCharsPerPage` to fit your context window, `skipDomains`, region and time range.
- **Search-only mode:** set `scrapePages: false` for SERP-only results (title, URL, snippet).
- **Pay per result:** $3 per 1,000 results.

### Use cases

- Give **ChatGPT / Claude / LangChain / LlamaIndex agents** live web access (works great over Apify MCP).
- **Grounding & fact-checking** answers with fresh sources.
- **Research automation:** collect and summarize everything written about a topic this week (`timeRange: "week"`).
- **Content & SEO research:** what the top-ranking pages actually say.

### Input

```json
{
  "queries": ["best vector database for RAG", "langgraph vs crewai"],
  "maxResultsPerQuery": 5,
  "scrapePages": true,
  "region": "us-en",
  "timeRange": "any",
  "maxCharsPerPage": 20000
}
```

### Output

#### Real output (default input)

First 2 of 5 records from a run with the prefilled input (5.5 s on a test machine; long values shortened here):

```json
[
  {
    "query": "best vector database for RAG",
    "rank": 1,
    "title": "We Tried and Tested 10 Best Vector Databases for RAG Pipelines",
    "url": "https://www.zenml.io/blog/vector-databases-for-rag",
    "displayedUrl": "www.zenml.io/blog/vector-databases-for-rag",
    "snippet": "For this article, we evaluated and tested the 10 best vector databases for RAG pipelines, assessing them under various parameters and compar…",
    "description": "Discover the 10 best data vector databases for RAG pipelines.",
    "author": "Hamza Tahir",
    "publishedAt": "2025-10-01T00:00:00.000Z",
    "modifiedAt": null,
    "siteName": null,
    "language": "en",
    "canonicalUrl": "https://www.zenml.io/blog/vector-databases-for-rag",
    "image": "https://assets.zenml.io/content/blog/vector-databases-for-rag/5e506905/vector-databases-for-rag-cover.jpg",
    "markdown": "The choice of Vector database for RAG pipelines can make or break your agent’s core paradigm.\n\nA well-chosen vector store recalls relevant d…",
    "wordCount": 4029,
    "extraction": "article",
    "httpStatus": 200,
    "error": null,
    "scrapedAt": "2026-10-15T12:11:03.278Z"
  },
  {
    "query": "best vector database for RAG",
    "rank": 2,
    "title": "Best vector databases for RAG in 2026",
    "url": "https://www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "displayedUrl": "www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "snippet": "Compare the top vector databases for RAG in 2026: Pinecone, Weaviate, Qdrant, Chroma, and Turbopuffer. Learn how each handles hosting, hybri…",
    "description": "Compare the top vector databases for RAG in 2026: Pinecone, Weaviate, Qdrant, Chroma, and Turbopuffer. Learn how each handles hosting, hybri…",
    "author": "Braintrust Team",
    "publishedAt": "11 July 2026",
    "modifiedAt": "11 July 2026",
    "siteName": "Braintrust",
    "language": "en",
    "canonicalUrl": "https://www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "image": "https://www.braintrust.dev/og?title=Best+vector+databases+for+RAG+in+2026&description=Compare+the+top+vector+databases+for+RAG+in+2026%3A+Pi…",
    "markdown": "TL;DR\n\nA vector database stores embeddings and returns the chunks a RAG app uses as context. When a user asks a question, the app embeds the…",
    "wordCount": 1970,
    "extraction": "article",
    "httpStatus": 200,
    "error": null,
    "scrapedAt": "2026-10-15T12:11:03.278Z"
  }
]
```

```json
{
  "query": "apify actor pricing pay per event",
  "rank": 1,
  "title": "Pay-per-event pricing model | Platform | Apify Documentation",
  "url": "https://docs.apify.com/actors/publishing/monetize/pay-per-event",
  "snippet": "The pay-per-event (PPE) pricing model offers a flexible monetization option…",
  "description": "Learn how to monetize your Actor with pay-per-event (PPE) pricing…",
  "language": "en",
  "markdown": "The pay-per-event (PPE) pricing model offers a flexible monetization option for Actors on Apify Store. The key benefits are:\n\n- Unlimited revenue scalability\n- AI/MCP compatibility…",
  "wordCount": 2459,
  "httpStatus": 200
}
```

### Pricing

- **$3.00 per 1,000 results** ($0.003 per result).
- The prefilled input returns 5 results → **$0.015**.
- Example: 1,000 results → $3.00; 10,000 → $30.00. Set **Max results (cost cap)** to limit any run.

* Failed or blocked pages are replaced by the next result and never charged.
* Your Apify spending limit is respected exactly: the actor stops charging and finishes cleanly when it is reached. Apify platform usage (compute, proxy) is included in the price.

### Use with AI agents (MCP), LangChain, RAG & the API

This actor is a ready-made **tool for AI agents**: an LLM can call it, wait for the run and read clean JSON back.

**MCP server (Claude Desktop, Claude Code, Cursor, VS Code, any MCP client)** — add Apify's hosted MCP server with this actor enabled as a tool:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=torix33/rag-web-search",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}
```

**LangChain / LangGraph** (`pip install langchain-apify`, set `APIFY_API_TOKEN`):

```python
from langchain_apify import ApifyActorsTool, ApifyWrapper
from langchain_core.documents import Document

## 1) give the actor to an agent as a tool
tool = ApifyActorsTool("torix33/rag-web-search")

## 2) or load results as Documents for a RAG index
loader = ApifyWrapper().call_actor(
    actor_id="torix33/rag-web-search",
    run_input={"queries": ["best vector database for RAG", "langgraph vs crewai"], "maxResultsPerQuery": 5, "scrapePages": True, "region": "us-en", "timeRange": "any", "maxCharsPerPage": 20000},
    dataset_mapping_function=lambda item: Document(page_content=item["markdown"], metadata={"url": item.get("url")}),
)
docs = loader.load()
```

**REST API — one synchronous call, dataset items in the response:**

```bash
curl -X POST "https://api.apify.com/v2/acts/torix33~rag-web-search/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["best vector database for RAG", "langgraph vs crewai"], "maxResultsPerQuery": 5, "scrapePages": true, "region": "us-en", "timeRange": "any", "maxCharsPerPage": 20000}'
```

Also works with the Apify Python/JS clients, **Make, Zapier, n8n**, Google Sheets, webhooks and schedules. Export as JSON, CSV, Excel, XML or RSS.

**RAG tip:** This actor *is* an agent tool: give the LLM the `markdown` of each result plus its `url` and ask it to answer with citations. Use `maxCharsPerPage` to fit your context window.

### Limitations

- Results come from DuckDuckGo (top 10) and are extended with Bing results when you ask for more. Duplicates are removed.
- JavaScript-only pages may return little text and are then replaced by the next result.

### FAQ

**Is it a Tavily, Exa or SerpAPI alternative?**
Yes, for grounding LLM answers: you get search results with page content as Markdown at $3 per 1,000 results.

**Which search engines are used?**
DuckDuckGo top 10, extended with Bing results when you ask for more; duplicates are removed.

**Can I get only titles, URLs and snippets?**
Yes. Set `scrapePages: false` for a SERP-only mode.

# Actor input Schema

## `queries` (type: `array`):

One web search per line.

## `maxResultsPerQuery` (type: `integer`):

How many top results to return for each query (up to ~30; DuckDuckGo first, then Bing).

## `scrapePages` (type: `boolean`):

On: open each result and return its main content as Markdown. Off: search results only (title, URL, snippet).

## `region` (type: `string`):

DuckDuckGo region code: <code>wt-wt</code> (worldwide), <code>us-en</code>, <code>uk-en</code>, <code>de-de</code>, <code>fr-fr</code>…

## `timeRange` (type: `string`):

Only results published within this period.

## `skipDomains` (type: `array`):

Results from these domains are skipped, e.g. <code>youtube.com</code>, <code>reddit.com</code>.

## `extractionMode` (type: `string`):

How page content is extracted.

## `minWordCount` (type: `integer`):

Thinner pages are replaced by the next result and not charged.

## `maxCharsPerPage` (type: `integer`):

Truncate Markdown to fit your LLM context. 0 = full page.

## `maxResults` (type: `integer`):

Stop after this many results in total. <code>0</code> = no limit.

## `maxConcurrency` (type: `integer`):

Pages fetched in parallel.

## `requestTimeoutSecs` (type: `integer`):

Slow pages are skipped and replaced by the next result.

## `proxyConfiguration` (type: `object`):

Datacenter proxy is used only for retries when a site or the search engine blocks.

## Actor input object example

```json
{
  "queries": [
    "best vector database for RAG"
  ],
  "maxResultsPerQuery": 5,
  "scrapePages": true,
  "region": "wt-wt",
  "timeRange": "any",
  "extractionMode": "article",
  "minWordCount": 50,
  "maxCharsPerPage": 0,
  "maxResults": 0,
  "maxConcurrency": 20,
  "requestTimeoutSecs": 10,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `runStats` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best vector database for RAG"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("torix33/rag-web-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["best vector database for RAG"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("torix33/rag-web-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best vector database for RAG"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call torix33/rag-web-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,torix33/rag-web-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/DC9RFpN4o7CfHphZH/builds/x9VmNnANQAQ2fXse6/openapi.json