RAG Web Search — Search Results as Markdown for LLMs & AI Agents

Give it a query, get back the top web results as clean Markdown — ready to drop into an LLM prompt, a RAG pipeline or an AI agent's tool call.

Every result contains the search rank, title, URL, snippet and (with page fetching on) the page's main content as Markdown (Readability: no menus, ads or cookie banners) plus description, author, published/modified dates, language and word count.

How to run it in 3 steps

1. Add your input. In Search queries paste search questions, one per line — e.g. best vector database for RAG. The form is already filled with a working example, so you can skip this step for a first try.
2. Press Start. The prefilled run returns 5 results in about 5 s and costs about $0.015. Set Max results (cost cap) to cap the cost of bigger runs; you only pay for results actually delivered.
3. Get the data. Open the Output tab and export JSON, CSV or Excel — or run it from the API, Make, Zapier, n8n or an AI agent (see below).

Features

• Fast: pure HTTP — 4 queries × 8 pages in ~6 seconds. No browser to boot.
• Always N results: pages that fail, block or are too thin are replaced by the next search result (and never charged).
• LLM-friendly controls: maxCharsPerPage to fit your context window, skipDomains, region and time range.
• Search-only mode: set scrapePages: false for SERP-only results (title, URL, snippet).
• Pay per result: $3 per 1,000 results.

Use cases

• Give ChatGPT / Claude / LangChain / LlamaIndex agents live web access (works great over Apify MCP).
• Grounding & fact-checking answers with fresh sources.
• Research automation: collect and summarize everything written about a topic this week (timeRange: "week").
• Content & SEO research: what the top-ranking pages actually say.

Input

{
  "queries": ["best vector database for RAG", "langgraph vs crewai"],
  "maxResultsPerQuery": 5,
  "scrapePages": true,
  "region": "us-en",
  "timeRange": "any",
  "maxCharsPerPage": 20000
}

Output

Real output (default input)

First 2 of 5 records from a run with the prefilled input (5.5 s on a test machine; long values shortened here):

[
  {
    "query": "best vector database for RAG",
    "rank": 1,
    "title": "We Tried and Tested 10 Best Vector Databases for RAG Pipelines",
    "url": "https://www.zenml.io/blog/vector-databases-for-rag",
    "displayedUrl": "www.zenml.io/blog/vector-databases-for-rag",
    "snippet": "For this article, we evaluated and tested the 10 best vector databases for RAG pipelines, assessing them under various parameters and compar…",
    "description": "Discover the 10 best data vector databases for RAG pipelines.",
    "author": "Hamza Tahir",
    "publishedAt": "2025-10-01T00:00:00.000Z",
    "modifiedAt": null,
    "siteName": null,
    "language": "en",
    "canonicalUrl": "https://www.zenml.io/blog/vector-databases-for-rag",
    "image": "https://assets.zenml.io/content/blog/vector-databases-for-rag/5e506905/vector-databases-for-rag-cover.jpg",
    "markdown": "The choice of Vector database for RAG pipelines can make or break your agent’s core paradigm.\n\nA well-chosen vector store recalls relevant d…",
    "wordCount": 4029,
    "extraction": "article",
    "httpStatus": 200,
    "error": null,
    "scrapedAt": "2026-10-15T12:11:03.278Z"
  },
  {
    "query": "best vector database for RAG",
    "rank": 2,
    "title": "Best vector databases for RAG in 2026",
    "url": "https://www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "displayedUrl": "www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "snippet": "Compare the top vector databases for RAG in 2026: Pinecone, Weaviate, Qdrant, Chroma, and Turbopuffer. Learn how each handles hosting, hybri…",
    "description": "Compare the top vector databases for RAG in 2026: Pinecone, Weaviate, Qdrant, Chroma, and Turbopuffer. Learn how each handles hosting, hybri…",
    "author": "Braintrust Team",
    "publishedAt": "11 July 2026",
    "modifiedAt": "11 July 2026",
    "siteName": "Braintrust",
    "language": "en",
    "canonicalUrl": "https://www.braintrust.dev/articles/best-vector-databases-for-rag-2026",
    "image": "https://www.braintrust.dev/og?title=Best+vector+databases+for+RAG+in+2026&description=Compare+the+top+vector+databases+for+RAG+in+2026%3A+Pi…",
    "markdown": "TL;DR\n\nA vector database stores embeddings and returns the chunks a RAG app uses as context. When a user asks a question, the app embeds the…",
    "wordCount": 1970,
    "extraction": "article",
    "httpStatus": 200,
    "error": null,
    "scrapedAt": "2026-10-15T12:11:03.278Z"
  }
]

{
  "query": "apify actor pricing pay per event",
  "rank": 1,
  "title": "Pay-per-event pricing model | Platform | Apify Documentation",
  "url": "https://docs.apify.com/actors/publishing/monetize/pay-per-event",
  "snippet": "The pay-per-event (PPE) pricing model offers a flexible monetization option…",
  "description": "Learn how to monetize your Actor with pay-per-event (PPE) pricing…",
  "language": "en",
  "markdown": "The pay-per-event (PPE) pricing model offers a flexible monetization option for Actors on Apify Store. The key benefits are:\n\n- Unlimited revenue scalability\n- AI/MCP compatibility…",
  "wordCount": 2459,
  "httpStatus": 200
}

Pricing

• $3.00 per 1,000 results ($0.003 per result).
• The prefilled input returns 5 results → $0.015.
• Example: 1,000 results → $3.00; 10,000 → $30.00. Set Max results (cost cap) to limit any run.

• Failed or blocked pages are replaced by the next result and never charged.
• Your Apify spending limit is respected exactly: the actor stops charging and finishes cleanly when it is reached. Apify platform usage (compute, proxy) is included in the price.

Use with AI agents (MCP), LangChain, RAG & the API

This actor is a ready-made tool for AI agents: an LLM can call it, wait for the run and read clean JSON back.

MCP server (Claude Desktop, Claude Code, Cursor, VS Code, any MCP client) — add Apify's hosted MCP server with this actor enabled as a tool:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=torix33/rag-web-search",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

LangChain / LangGraph (pip install langchain-apify, set APIFY_API_TOKEN):

from langchain_apify import ApifyActorsTool, ApifyWrapper
from langchain_core.documents import Document

# 1) give the actor to an agent as a tool
tool = ApifyActorsTool("torix33/rag-web-search")

# 2) or load results as Documents for a RAG index
loader = ApifyWrapper().call_actor(
    actor_id="torix33/rag-web-search",
    run_input={"queries": ["best vector database for RAG", "langgraph vs crewai"], "maxResultsPerQuery": 5, "scrapePages": True, "region": "us-en", "timeRange": "any", "maxCharsPerPage": 20000},
    dataset_mapping_function=lambda item: Document(page_content=item["markdown"], metadata={"url": item.get("url")}),
)
docs = loader.load()

REST API — one synchronous call, dataset items in the response:

curl -X POST "https://api.apify.com/v2/acts/torix33~rag-web-search/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["best vector database for RAG", "langgraph vs crewai"], "maxResultsPerQuery": 5, "scrapePages": true, "region": "us-en", "timeRange": "any", "maxCharsPerPage": 20000}'

Also works with the Apify Python/JS clients, Make, Zapier, n8n, Google Sheets, webhooks and schedules. Export as JSON, CSV, Excel, XML or RSS.

RAG tip: This actor is an agent tool: give the LLM the markdown of each result plus its url and ask it to answer with citations. Use maxCharsPerPage to fit your context window.

Limitations

• Results come from DuckDuckGo (top 10) and are extended with Bing results when you ask for more. Duplicates are removed.
• JavaScript-only pages may return little text and are then replaced by the next result.

FAQ

Is it a Tavily, Exa or SerpAPI alternative? Yes, for grounding LLM answers: you get search results with page content as Markdown at $3 per 1,000 results.

Which search engines are used? DuckDuckGo top 10, extended with Bing results when you ask for more; duplicates are removed.

Can I get only titles, URLs and snippets? Yes. Set scrapePages: false for a SERP-only mode.