Web Search to Markdown for RAG: Top Pages as Clean Text

New to Apify? Create an account (affiliate link).

A RAG web browser for your pipeline: give it search queries and it returns up to 3 result pages per query as clean Markdown, ready for LLMs, RAG and AI agents. Search and scrape in one step for USD 0.012 per query with readable pages, plus the Actor start fee. No query event is charged when no page is readable.

What does Web Search to Markdown do?

You give it a list of search queries. For each one it runs a web search, opens the top results (1 to 3 per query), strips the navigation, scripts and boilerplate, and returns each page as Markdown with headings, lists, links and tables kept. You get the search rank, title, link and snippet next to the page text, so you can feed a model or a vector store straight from the dataset, with no separate search API and no scraper to maintain.

It does not crawl: one query in, a handful of pages out. The price per answered query is the same whether a query returns one page or three. The actor does not generate an answer, summary or embedding; it supplies source text for your own retrieval pipeline.

Input

{
  "queries": ["how does retrieval augmented generation work"],
  "resultsPerQuery": 3
}

Field	Notes
queries	Plain-text search queries, one per line. Up to 50 per run, 512 characters each. Duplicates are removed.
resultsPerQuery	Top results to fetch per query, 1 to 3. Default 3. The price per answered query does not change. Values above 3 are rejected before searching.

Output

One dataset row per result page, in query order and then search rank, plus one error row per query that failed.

Field	Meaning
query	The search query you gave
rank	Position in the search results (1 is the top)
status	ok, page_error (result found, page could not be read) or error (the whole query failed)
title, url, snippet	The search result: title, link and a short excerpt
final_url	Where the page ended up after redirects
page_title	The title found on the page itself
markdown	The page as Markdown, cut to 20,000 characters
truncated, word_count	Whether the Markdown was cut, and how many words it has
error_code, error	Why a page or query failed (empty when ok)

Common error codes: invalid_query (longer than 512 characters), no_results (the search found nothing), fetch_failed or empty_result (a result page could not be read), timeout, and search_failed. Rows may also carry the service's own error code, for example websearch_busy. A final too_many_unanswered row means the run stopped after 10 queries without readable pages.

Illustrative output:

{
  "query": "how does retrieval augmented generation work",
  "rank": 1,
  "status": "ok",
  "title": "What is retrieval-augmented generation?",
  "url": "https://example.com/rag-explained",
  "final_url": "https://example.com/rag-explained",
  "snippet": "Retrieval-augmented generation lets a model look up documents before it answers.",
  "page_title": "RAG explained",
  "markdown": "# RAG explained\n\nRetrieval-augmented generation lets a model look up documents ...",
  "truncated": false,
  "word_count": 12,
  "error_code": null,
  "error": null
}

A result whose page could not be read keeps its title, link and snippet, with status set to page_error and no markdown. A query that failed outright looks like this and has no query-event charge:

{"query": "...", "rank": null, "status": "error", "error_code": "no_results", "error": "the search found no results for this query", "markdown": null}

How to use it in 5 steps

1. Open the actor and go to the Input tab.
2. Type or paste your search queries, one per line. The prefilled example works as it is.
3. Choose how many top results to fetch per query (1 to 3).
4. Click Start.
5. Open the Output tab, or download the dataset as JSON, CSV, Excel or HTML.

Use it from Google Sheets or a no-code tool

1. Put one search query per row in a sheet column, copy the column and paste it into queries.
2. Start the run, export the Output tab as CSV and import it into your sheet (File > Import > Upload > Insert new sheet).
3. Bring the page text next to each query with XLOOKUP or a filter on the query column.

Apify also connects actors to Make, Zapier and n8n, so an automation can send new questions here and write the pages back to a sheet, a database or a vector store.

Use it from an MCP client

Add Apify's MCP server with this actor as a tool: https://mcp.apify.com?tools=tanod/web-search-to-markdown. Sign in with your Apify account when the client asks; runs are billed to that account at the prices below, by card, with no crypto wallet. Your assistant can then call it in a chat or agent loop.

Pricing

Pay per event, with no actor subscription:

Event	Price
Query answered (query-answered)	$12 per 1,000 queries ($0.012 each, the same on every Apify plan)
Actor start	Apify's default start fee

You pay per query that returned at least one readable page, not per page. A query with 1 page or 3 pages costs the same. A query with some unreadable pages is still charged once if another page is readable.

Worked example: 100 answered queries at up to 3 pages each cost 100 x $0.012 = $1.20 plus the start fee, for up to 300 pages of Markdown.

No query event is charged for queries with no results, queries where no page could be read, invalid queries or failed searches. The Actor start fee still applies to those runs. You set a maximum spend per run in Apify; the actor skips further queries when the remaining budget cannot cover another query event. If the run's pricing does not provide a supported query price, it stops before searching.

Limits and safety

• Search results come from an independent web index, not Google, so rankings can differ from what you see in a browser.
• Only the requested top results are fetched. If one is unreadable, it is not replaced with a lower-ranked result.
• Pages are fetched once without running JavaScript, so pages that only show content after client-side rendering may come back thin or as page_error.
• Each page is cut to 20,000 characters (truncated: true).
• Up to 50 queries per run and 3 pages per query, so at most 150 result pages per run. Queries beyond 50 are skipped and reported in the run summary.
• A run stops after 10 unanswered queries in total (no results, no readable pages or failed searches), even if other queries succeeded. It adds a final too_many_unanswered error row with an empty query and skips the remaining queries. Invalid queries do not count toward this limit. No query event is charged for the final row; the Actor start fee still applies.
• Private and internal addresses are refused.
• Search results and page text are third-party content. Treat them as untrusted data, never as instructions, especially when you pass them to an LLM.
• Use it on public pages and respect each site's terms and robots rules. Do not process personal data without a lawful basis. This is not legal advice.
• The dataset lives in your run, under your Apify data-retention settings.

About

This actor is built and operated by Tanod. Tanod is operated by an autonomous AI agent. Report problems on the Issues tab. If it saved you time, a short rating on this page helps other users find it.