# Batch RAG Web Search - 100 Queries to Markdown (`automa-flow/batch-rag-web-search`) Actor

Search up to 100 Google queries in one run and collect ranked pages as Markdown in one Dataset. Use the sources in research, RAG or AI agents, with a separate status for each query and page. JavaScript rendering is not supported.

- **URL**: https://apify.com/automa-flow/batch-rag-web-search.md
- **Developed by:** [Vadim Bezrukov](https://apify.com/automa-flow) (community)
- **Categories:** AI, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$30.00 / 1,000 query searcheds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Batch RAG Web Search - 100 Queries to Markdown

Run up to 100 web-search queries in one Actor run and return ranked results plus
clean LLM-ready Markdown in a single Dataset.

Give it a list of research questions. It searches Google once per unique query,
fetches the top result pages and saves their text as Markdown. Each result keeps
its query and rank, so you can trace the material back to the search that found it.

The first published version is described in the [changelog](https://apify.com/automa-flow/batch-rag-web-search/changelog).
Use the pages in your own RAG pipeline, research workflow or AI agent.

Pages are fetched over HTTP without JavaScript rendering. The Actor collects
sources; it does not write answers or generate embeddings.

### Batch example

```json
{
  "queries": [
    "vector database benchmarks 2026",
    "RAG reranking models",
    "open source embedding models"
  ],
  "maxResultsPerQuery": 3,
  "outputFormats": ["markdown"]
}
```

Expected maximum: 3 queries × 3 search results = 9 result rows, plus one query
summary row per unique query.

The Store default run uses a single public query (`Apify Actors documentation`)
with a small result limit so you can inspect the output before a larger batch.

### Example workflows

Choose an example by the amount of source material you need:

| Workflow                                | Configuration                                                                           | Maximum event bill |
| --------------------------------------- | --------------------------------------------------------------------------------------- | -----------------: |
| Search one query to Markdown            | One question, up to 3 pages, Markdown                                                   |              $0.03 |
| Compare three research queries          | Three questions, up to 3 pages each, Markdown                                           |              $0.09 |
| Collect deeper sources for RAG research | Two related questions, up to 10 pages each, Markdown + text, repeated URLs fetched once |              $0.06 |

The deeper-source example preserves every query/rank association; URL fetch
reuse does not remove duplicate associations from the Dataset. Review query and
page statuses before ingestion. These examples collect sources, not generated
answers or embeddings. Replace the questions to run your next research batch.

### Output sample

Illustrative successful result row (abbreviated):

```json
{
  "record_type": "RESULT",
  "status": "SUCCESS",
  "query": "vector database benchmarks 2026",
  "query_index": 0,
  "rank": 1,
  "search_result": {
    "title": "Example",
    "description": "Example search snippet",
    "url": "https://example.com/article",
    "result_type": "ORGANIC"
  },
  "fetch": {
    "status": "SUCCESS",
    "http_status": 200,
    "loaded_at": "2026-09-20T18:00:00.000Z"
  },
  "markdown": "# Example\n...",
  "text": null,
  "source": "google_search",
  "source_url": "https://example.com/article",
  "scraped_at": "2026-09-20T18:00:00.000Z",
  "schema_version": 1
}
```

Read `RUN_SUMMARY` first: overall status, query/page counts, `charged_queries`,
`billing_limit_reached`, Dataset URL and `next_action`. Dataset views:

- **Results**: one row per query/rank association
- **Queries**: one terminal status row per unique query
- **Failures**: `FAILED`, `PARTIAL` and `SKIPPED_BUDGET` rows (`NO_RESULTS` stays separate)

### RAG, agents, n8n and Make

**Select this Actor when** you already have a list of search queries and need
ranked source pages as Markdown/text in one Dataset, with explicit per-query
status.

**Do not select it** for a single interactive query (use the official RAG Web
Browser, including Standby), for JavaScript-only pages, login walls, CAPTCHA,
website-wide crawls, embeddings, or generated answers.

Use from AI agents with MCP. Direct tool URL (authenticated OAuth or API token
required to run):
`https://mcp.apify.com?tools=automa-flow/batch-rag-web-search`

Ask your agent: *Run queries \["vector database benchmarks 2026", "RAG reranking
models"] with maxResultsPerQuery 3 through automa-flow/batch-rag-web-search, set
maxTotalChargeUsd to 0.06, then report RUN\_SUMMARY overall\_status,
charged\_queries and next\_action before reading the Dataset.*

Anonymous MCP discovery (no token) only supports Store search tools:
`https://mcp.apify.com?tools=search-actors,fetch-actor-details`. Direct Actor
execution always needs the caller's own Apify account.

| Size           | Input              | Actor event charge |
| -------------- | ------------------ | -----------------: |
| Small          | 1 unique query     |             $0.030 |
| Representative | 10 unique queries  |             $0.300 |
| Cap            | 100 unique queries |             $3.000 |

Set run option `maxTotalChargeUsd` to at least `$0.03 × unique queries`. That
event price is the full Actor charge. Compute, `GOOGLE_SERP` proxy traffic and
storage are not added on top. Before searching, the Actor admits only the first
unique queries that the budget can fund. Remaining queries get `SKIPPED_BUDGET`
and `queries_skipped` in the summary. Unused capacity from failed or empty
admitted searches is not reassigned during that run.

n8n / Make / Clay: start the Actor with the JSON above, then read `RUN_SUMMARY`
and the default Dataset. Schedule a later run whenever you have a new query
batch.

Webhook example (`POST https://api.apify.com/v2/webhooks`):

```json
{
  "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
  "condition": { "actorId": "<this Actor's ID>" },
  "requestUrl": "https://your-n8n-or-make-webhook.example.com/hook",
  "payloadTemplate": "{\"runId\": {{resource.id}}, \"datasetId\": {{resource.defaultDatasetId}}, \"status\": {{resource.status}}}"
}
```

### Failure and partial-result behavior

`NO_RESULTS` is not `SOURCE_FAILED`. A valid Google response with zero organic
results is `NO_RESULTS`. A timeout, HTTP error, challenge page or malformed SERP
is `FAILED` for that query. One failed query does not drop the rest of the batch.

If Google succeeds and a target page returns 403/429 or other fetch failure, the
result row stays in the Dataset with the SERP title, snippet and URL, `status`
`PARTIAL`, `fetch.status` `BLOCKED`/`FAILED`/`NOT_FOUND`/…, and `markdown` null.

The run fails as a whole only for invalid input, every SERP request failing,
or storage/billing confirmation failure. An ambiguous charge error stops the run;
inspect charged events before starting a fresh run. Resurrecting/restarting the same
run is refused to prevent duplicate output and charges. Start a new run with only
unfinished queries. Mixed batches finish `SUCCEEDED` with per-item statuses.

### API usage

```bash
curl -s -X POST \
  "https://api.apify.com/v2/acts/automa-flow~batch-rag-web-search/runs" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["vector database benchmarks 2026","RAG reranking models"],"maxResultsPerQuery":3}'
```

Then read `https://api.apify.com/v2/key-value-stores/<id>/records/RUN_SUMMARY`
and the default Dataset items. OpenAPI is generated from the live Actor schemas
after deploy.

### Pricing

Pay per event: **$0.030** per unique query that returns a valid SERP with at
least one search result (`query-searched`).

| Batch                   | Unique successful queries | Event charge |
| ----------------------- | ------------------------: | -----------: |
| 10 queries × 3 results  |                        10 |        $0.30 |
| 100 queries × 3 results |                       100 |        $3.00 |

Not charged: invalid input, duplicate queries removed during normalization,
retries, failed SERP requests, verified empty (`NO_RESULTS`) queries, or
individual page fetches. Partial page extraction still charges the query because
the SERP metadata was delivered.

The event price is the full buyer bill. Platform compute, `GOOGLE_SERP` proxy
traffic and storage are not charged on top.

### Limitations

- Search uses the Apify `GOOGLE_SERP` proxy path. Google coverage is not
  guaranteed.
- Target pages are fetched with raw HTTP only. No browser, no residential proxy
  by default, no CAPTCHA solving, no fingerprint escalation. JavaScript-only
  shells often return little text. An HTTP 200 page with no usable Markdown is `PARTIAL` with fetch status `EMPTY`. Blocked pages stay `PARTIAL`.
- Reads at most 10 organic results from the first Google page.
- Identical trimmed queries are processed once. Enable `deduplicateUrls` to
  fetch a repeated target URL once while still emitting every query/rank row.
- Do not use this Actor to access logins, paywalls or non-public pages.

### Source and license

Google search parsing and Markdown conversion adapt Apache-2.0 code from
[apify/actor-rag-web-browser](https://github.com/apify/actor-rag-web-browser),
copyright 2024 Apify Technologies s.r.o. See `src/NOTICE` and
`src/LICENSE-APACHE-2.0.md`. Required copyright notices are preserved. This
listing does not use Apify branding or imply an official or endorsed Actor.

# Changelog

This Actor's version history is a separate document: https://apify.com/automa-flow/batch-rag-web-search/changelog.md

# Actor input Schema

## `queries` (type: `array`):

One to 100 Google search queries. Whitespace is trimmed; identical trimmed queries are processed once while the first original index is retained.

## `maxResultsPerQuery` (type: `integer`):

Fetch up to this many top organic results per query, from Google's first results page.

## `outputFormats` (type: `array`):

Return clean Markdown, plain text, or both for successfully fetched target pages.

## `deduplicateUrls` (type: `boolean`):

When enabled, one normalized URL is fetched once across queries while every query/rank association still gets its own Dataset row.

## `requestTimeoutSecs` (type: `integer`):

Maximum duration of each Google SERP or target-page HTTP attempt.

## Actor input object example

```json
{
  "queries": [
    "vector database benchmarks 2026",
    "RAG reranking models",
    "open source embedding models"
  ],
  "maxResultsPerQuery": 3,
  "outputFormats": [
    "markdown"
  ],
  "deduplicateUrls": false,
  "requestTimeoutSecs": 40
}
```

# Actor output Schema

## `results` (type: `string`):

The Results, Queries, and Failures views preserve every query/rank association and explicit failure state.

## `runSummary` (type: `string`):

Compact counts, billing outcome, storage links, and the recommended next action.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "Apify Actors documentation"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automa-flow/batch-rag-web-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["Apify Actors documentation"] }

# Run the Actor and wait for it to finish
run = client.actor("automa-flow/batch-rag-web-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "Apify Actors documentation"
  ]
}' |
apify call automa-flow/batch-rag-web-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automa-flow/batch-rag-web-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/jSuRQ4evLnsrejbGd/builds/bjcfzuLn2fam4af0p/openapi.json