Batch RAG Web Search - 100 Queries to Markdown

Run up to 100 web-search queries in one Actor run and return ranked results plus clean LLM-ready Markdown in a single Dataset.

This Actor is a bounded batch retrieval primitive for AI agents, RAG pipelines, n8n/Make workflows, research jobs and backend enrichment. It searches Google once per unique query, fetches the top result pages over raw HTTP, and writes one Dataset row per query/rank association. It does not synthesize answers.

It is not the official apify/rag-web-browser Actor and is not affiliated with or endorsed by Apify beyond ordinary platform use.

Batch example

{
  "queries": [
    "vector database benchmarks 2026",
    "RAG reranking models",
    "open source embedding models"
  ],
  "maxResultsPerQuery": 3,
  "outputFormats": ["markdown"]
}

Expected maximum: 3 queries × 3 search results = 9 result rows, plus one query summary row per unique query.

The Store default run uses a single public query (Apify Actors documentation) so the first run can finish with a non-empty Dataset.

Example workflows

Choose an example by the amount of source material you need:

Workflow	Configuration	Maximum event bill
Search one query to Markdown	One question, up to 3 pages, Markdown	$0.03
Compare three research queries	Three questions, up to 3 pages each, Markdown	$0.09
Collect deeper sources for RAG research	Two related questions, up to 10 pages each, Markdown + text, repeated URLs fetched once	$0.06

The deeper-source example preserves every query/rank association; URL fetch reuse does not remove duplicate associations from the Dataset. Review query and page statuses before ingestion. These examples collect sources, not generated answers or embeddings. Replace the questions to run your next research batch.

Output sample

One successful result row (abbreviated):

{
  "record_type": "RESULT",
  "status": "SUCCESS",
  "query": "vector database benchmarks 2026",
  "query_index": 0,
  "rank": 1,
  "search_result": {
    "title": "Example",
    "description": "Example search snippet",
    "url": "https://example.com/article",
    "result_type": "ORGANIC"
  },
  "fetch": {
    "status": "SUCCESS",
    "http_status": 200,
    "loaded_at": "2026-09-20T18:00:00.000Z"
  },
  "markdown": "# Example\n...",
  "text": null,
  "source": "google_search",
  "source_url": "https://example.com/article",
  "scraped_at": "2026-09-20T18:00:00.000Z",
  "schema_version": 1
}

Read RUN_SUMMARY first: overall status, query/page counts, charged_queries, billing_limit_reached, Dataset URL and next_action. Dataset views:

• Results: one row per query/rank association
• Queries: one terminal status row per unique query
• Failures: FAILED, PARTIAL, SKIPPED_BUDGET and SKIPPED_TIME_LIMIT rows (NO_RESULTS stays separate)

RAG, agents, n8n and Make

Select this Actor when you already have a list of search queries and need ranked source pages as Markdown/text in one Dataset, with explicit per-query status.

Do not select it for a single interactive query (use the official RAG Web Browser, including Standby), for JavaScript-only pages, login walls, CAPTCHA, website-wide crawls, embeddings, or generated answers.

Use from AI agents with MCP. Direct tool URL (authenticated OAuth or API token required to run): https://mcp.apify.com?tools=automa-flow/batch-rag-web-search

Ask your agent: Run queries ["vector database benchmarks 2026", "RAG reranking models"] with maxResultsPerQuery 3 through automa-flow/batch-rag-web-search, set maxTotalChargeUsd to 0.06, then report RUN_SUMMARY overall_status, charged_queries and next_action before reading the Dataset.

Anonymous MCP discovery (no token) only supports Store search tools: https://mcp.apify.com?tools=search-actors,fetch-actor-details. Direct Actor execution always needs the caller's own Apify account.

Size	Input	Actor event charge
Small	1 unique query	$0.030
Representative	10 unique queries	$0.300
Cap	100 unique queries	$3.000

Set run option maxTotalChargeUsd to at least $0.03 × unique queries. That event price is the full Actor charge. Compute, GOOGLE_SERP proxy traffic and storage are not added on top. Before searching, the Actor admits only the first unique queries that the budget can fund. Remaining queries get SKIPPED_BUDGET and queries_skipped in the summary. Unused capacity from failed or empty admitted searches is not reassigned during that run.

Give large batches a run timeout that fits them. Near the timeout, the Actor stops starting new searches and page fetches and keeps a minute to charge and write the summary. Queries it did not search get SKIPPED_TIME_LIMIT and are not charged; pages it did not fetch get fetch.status TIMEOUT with error.code RUN_DEADLINE. RUN_SUMMARY.time_limit_reached is then true.

n8n / Make / Clay: start the Actor with the JSON above, then read RUN_SUMMARY and the default Dataset. Schedule a later run whenever you have a new query batch.

Webhook example (POST https://api.apify.com/v2/webhooks):

{
  "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
  "condition": { "actorId": "<this Actor's ID>" },
  "requestUrl": "https://your-n8n-or-make-webhook.example.com/hook",
  "payloadTemplate": "{\"runId\": {{resource.id}}, \"datasetId\": {{resource.defaultDatasetId}}, \"status\": {{resource.status}}}"
}

Failure and partial-result behavior

NO_RESULTS is not SOURCE_FAILED. A valid Google response with zero organic results is NO_RESULTS. A timeout, HTTP error, challenge page or malformed SERP is FAILED for that query. One failed query does not drop the rest of the batch.

If Google succeeds and a target page returns 403/429 or other fetch failure, the result row stays in the Dataset with the SERP title, snippet and URL, status PARTIAL, fetch.status BLOCKED/FAILED/NOT_FOUND/…, and markdown null.

The run fails as a whole only for invalid input, every SERP request failing, or storage/billing confirmation failure. An ambiguous charge error stops the run; inspect charged events before starting a fresh run. Resurrecting/restarting the same run is refused to prevent duplicate output and charges. Start a new run with only unfinished queries. Mixed batches finish SUCCEEDED with per-item statuses.

API usage

curl -s -X POST \
  "https://api.apify.com/v2/acts/automa-flow~batch-rag-web-search/runs" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["vector database benchmarks 2026","RAG reranking models"],"maxResultsPerQuery":3}'

Then read https://api.apify.com/v2/key-value-stores/<id>/records/RUN_SUMMARY and the default Dataset items. OpenAPI is generated from the live Actor schemas after deploy.

Related automa-flow tools

• Ingest a whole site instead of search results: Website Content Crawler turns up to 10 start URLs into Markdown and chunks. It is for sites you own or are authorized to process, and each source needs a rights declaration.
• Extract typed fields instead of Markdown: AI Schema Web Extractor returns JSON that matches your schema for the result URLs you select.
• Send the context to a model: OpenRouter for Agents takes chat messages, tool definitions and fallback models, and returns structured output. Pass the Markdown rows you keep as message content.

Each Actor runs and charges separately. This Actor does not call them.

Pricing

Pay per event: $0.030 per unique query that returns a valid SERP with at least one search result (query-searched).

Batch	Unique successful queries	Event charge
10 queries × 3 results	10	$0.30
100 queries × 3 results	100	$3.00

Not charged: invalid input, duplicate queries removed during normalization, retries, failed SERP requests, verified empty (NO_RESULTS) queries, or individual page fetches. Partial page extraction still charges the query because the SERP metadata was delivered.

The event price is the full buyer bill. Platform compute, GOOGLE_SERP proxy traffic and storage are not charged on top. A measured 100-query staging run used about $0.003 of those resources per charged query.

Limitations

• Normal Actor runs of the official RAG Web Browser still take one query; that Actor can process requests concurrently in Standby. This Actor's wedge is a bounded queries[] batch that writes one Dataset.
• Search uses the Apify GOOGLE_SERP proxy path. Google coverage is not guaranteed.
• Target pages are fetched with raw HTTP only. No browser, no residential proxy by default, no CAPTCHA solving, no fingerprint escalation. JavaScript-only shells often return little text. An HTTP 200 page with no usable Markdown is PARTIAL with fetch status EMPTY. Blocked pages stay PARTIAL.
• MVP reads at most 10 organic results from the first Google page.
• Identical trimmed queries are processed once. Enable deduplicateUrls to fetch a repeated target URL once while still emitting every query/rank row.
• Do not use this Actor to access logins, paywalls or non-public pages.

Source and license

Google search parsing and Markdown conversion adapt Apache-2.0 code from apify/actor-rag-web-browser, copyright 2024 Apify Technologies s.r.o. See src/NOTICE and src/LICENSE-APACHE-2.0.md. Required copyright notices are preserved. This listing does not use Apify branding or imply an official or endorsed Actor.

Technical implementation

queries[] → unique trimmed queries
  → bounded Google SERP (GOOGLE_SERP proxy, raw HTTP)
  → top N organic URLs
  → bounded raw-HTTP page fetch
  → Markdown/text
  → Dataset RESULT rows + QUERY rows + RUN_SUMMARY

Query concurrency is 4 and page-fetch concurrency is 12. Internal retry and concurrency knobs are not Store inputs. No Playwright, Puppeteer, Camoufox, LLM, or cross-run state. A run-local KVS marker prevents unsafe replay.