Batch RAG Web Search - 100 Queries to Markdown

Run up to 100 web-search queries in one Actor run and return ranked results plus clean LLM-ready Markdown in a single Dataset.

Give it a list of research questions. It searches Google once per unique query, fetches the top result pages and saves their text as Markdown. Each result keeps its query and rank, so you can trace the material back to the search that found it.

The first published version is described in the changelog. Use the pages in your own RAG pipeline, research workflow or AI agent.

Pages are fetched over HTTP without JavaScript rendering. The Actor collects sources; it does not write answers or generate embeddings.

Batch example

{
  "queries": [
    "vector database benchmarks 2026",
    "RAG reranking models",
    "open source embedding models"
  ],
  "maxResultsPerQuery": 3,
  "outputFormats": ["markdown"]
}

Expected maximum: 3 queries × 3 search results = 9 result rows, plus one query summary row per unique query.

The Store default run uses a single public query (Apify Actors documentation) with a small result limit so you can inspect the output before a larger batch.

Example workflows

Choose an example by the amount of source material you need:

Workflow	Configuration	Maximum event bill
Search one query to Markdown	One question, up to 3 pages, Markdown	$0.03
Compare three research queries	Three questions, up to 3 pages each, Markdown	$0.09
Collect deeper sources for RAG research	Two related questions, up to 10 pages each, Markdown + text, repeated URLs fetched once	$0.06

The deeper-source example preserves every query/rank association; URL fetch reuse does not remove duplicate associations from the Dataset. Review query and page statuses before ingestion. These examples collect sources, not generated answers or embeddings. Replace the questions to run your next research batch.

Output sample

Illustrative successful result row (abbreviated):

{
  "record_type": "RESULT",
  "status": "SUCCESS",
  "query": "vector database benchmarks 2026",
  "query_index": 0,
  "rank": 1,
  "search_result": {
    "title": "Example",
    "description": "Example search snippet",
    "url": "https://example.com/article",
    "result_type": "ORGANIC"
  },
  "fetch": {
    "status": "SUCCESS",
    "http_status": 200,
    "loaded_at": "2026-09-20T18:00:00.000Z"
  },
  "markdown": "# Example\n...",
  "text": null,
  "source": "google_search",
  "source_url": "https://example.com/article",
  "scraped_at": "2026-09-20T18:00:00.000Z",
  "schema_version": 1
}

Read RUN_SUMMARY first: overall status, query/page counts, charged_queries, billing_limit_reached, Dataset URL and next_action. Dataset views:

• Results: one row per query/rank association
• Queries: one terminal status row per unique query
• Failures: FAILED, PARTIAL and SKIPPED_BUDGET rows (NO_RESULTS stays separate)

RAG, agents, n8n and Make

Select this Actor when you already have a list of search queries and need ranked source pages as Markdown/text in one Dataset, with explicit per-query status.

Do not select it for a single interactive query (use the official RAG Web Browser, including Standby), for JavaScript-only pages, login walls, CAPTCHA, website-wide crawls, embeddings, or generated answers.

Use from AI agents with MCP. Direct tool URL (authenticated OAuth or API token required to run): https://mcp.apify.com?tools=automa-flow/batch-rag-web-search

Ask your agent: Run queries ["vector database benchmarks 2026", "RAG reranking models"] with maxResultsPerQuery 3 through automa-flow/batch-rag-web-search, set maxTotalChargeUsd to 0.06, then report RUN_SUMMARY overall_status, charged_queries and next_action before reading the Dataset.

Anonymous MCP discovery (no token) only supports Store search tools: https://mcp.apify.com?tools=search-actors,fetch-actor-details. Direct Actor execution always needs the caller's own Apify account.

Size	Input	Actor event charge
Small	1 unique query	$0.030
Representative	10 unique queries	$0.300
Cap	100 unique queries	$3.000

Set run option maxTotalChargeUsd to at least $0.03 × unique queries. That event price is the full Actor charge. Compute, GOOGLE_SERP proxy traffic and storage are not added on top. Before searching, the Actor admits only the first unique queries that the budget can fund. Remaining queries get SKIPPED_BUDGET and queries_skipped in the summary. Unused capacity from failed or empty admitted searches is not reassigned during that run.

n8n / Make / Clay: start the Actor with the JSON above, then read RUN_SUMMARY and the default Dataset. Schedule a later run whenever you have a new query batch.

Webhook example (POST https://api.apify.com/v2/webhooks):

{
  "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
  "condition": { "actorId": "<this Actor's ID>" },
  "requestUrl": "https://your-n8n-or-make-webhook.example.com/hook",
  "payloadTemplate": "{\"runId\": {{resource.id}}, \"datasetId\": {{resource.defaultDatasetId}}, \"status\": {{resource.status}}}"
}

Failure and partial-result behavior

NO_RESULTS is not SOURCE_FAILED. A valid Google response with zero organic results is NO_RESULTS. A timeout, HTTP error, challenge page or malformed SERP is FAILED for that query. One failed query does not drop the rest of the batch.

If Google succeeds and a target page returns 403/429 or other fetch failure, the result row stays in the Dataset with the SERP title, snippet and URL, status PARTIAL, fetch.status BLOCKED/FAILED/NOT_FOUND/…, and markdown null.

The run fails as a whole only for invalid input, every SERP request failing, or storage/billing confirmation failure. An ambiguous charge error stops the run; inspect charged events before starting a fresh run. Resurrecting/restarting the same run is refused to prevent duplicate output and charges. Start a new run with only unfinished queries. Mixed batches finish SUCCEEDED with per-item statuses.

API usage

curl -s -X POST \
  "https://api.apify.com/v2/acts/automa-flow~batch-rag-web-search/runs" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["vector database benchmarks 2026","RAG reranking models"],"maxResultsPerQuery":3}'

Then read https://api.apify.com/v2/key-value-stores/<id>/records/RUN_SUMMARY and the default Dataset items. OpenAPI is generated from the live Actor schemas after deploy.

Pricing

Pay per event: $0.030 per unique query that returns a valid SERP with at least one search result (query-searched).

Batch	Unique successful queries	Event charge
10 queries × 3 results	10	$0.30
100 queries × 3 results	100	$3.00

Not charged: invalid input, duplicate queries removed during normalization, retries, failed SERP requests, verified empty (NO_RESULTS) queries, or individual page fetches. Partial page extraction still charges the query because the SERP metadata was delivered.

The event price is the full buyer bill. Platform compute, GOOGLE_SERP proxy traffic and storage are not charged on top.

Limitations

• Search uses the Apify GOOGLE_SERP proxy path. Google coverage is not guaranteed.
• Target pages are fetched with raw HTTP only. No browser, no residential proxy by default, no CAPTCHA solving, no fingerprint escalation. JavaScript-only shells often return little text. An HTTP 200 page with no usable Markdown is PARTIAL with fetch status EMPTY. Blocked pages stay PARTIAL.
• Reads at most 10 organic results from the first Google page.
• Identical trimmed queries are processed once. Enable deduplicateUrls to fetch a repeated target URL once while still emitting every query/rank row.
• Do not use this Actor to access logins, paywalls or non-public pages.

Source and license

Google search parsing and Markdown conversion adapt Apache-2.0 code from apify/actor-rag-web-browser, copyright 2024 Apify Technologies s.r.o. See src/NOTICE and src/LICENSE-APACHE-2.0.md. Required copyright notices are preserved. This listing does not use Apify branding or imply an official or endorsed Actor.