RAG Web Search - Results as Markdown for AI

Give your AI agent or RAG pipeline live web knowledge in one call. Send a search query and the Actor returns the top web results with title, URL and snippet, and opens each result to give you the page as clean Markdown: main content only, with title, author, publish date and language. Ready to drop into an LLM prompt, a vector database or an agent tool.

No browser, no search API key: the Actor searches the web through DuckDuckGo and converts each page with a fast HTML-to-Markdown engine. You pay $0.001 per result, and results whose page cannot be read are free.

What you get

One Dataset row per result:

• position, title, url, displayedUrl, snippet: the search result.
• markdown: the page as clean Markdown (headings, lists, tables, code blocks, absolute links), main content only by default.
• pageTitle, description, author, publishedAt, language, wordCount.
• truncated when the page was cut at Max characters per page, likelyJsRendered when the page needs JavaScript and little text was found.
• error when a page could not be fetched (for example a site that blocks automated visits): that row keeps the search data and is not charged.
• query, region, and isNew on scheduled runs (URL not in the previous results).

How to use it

1. Open the Input tab.
2. In Search queries type one per line, for example best open source CRM for startups. Search operators such as site: and quotes work.
3. Set Results per query (default 5, up to 30), Region and, optionally, Published within.
4. Keep Fetch pages as Markdown on for full content, or switch it off to get only titles, URLs and snippets.
5. Click Start and read the rows, or call it from your agent through the API or the Apify MCP server.

Input

{
  "queries": ["best open source CRM for startups"],
  "maxResults": 5,
  "scrapePages": true,
  "region": "us-en",
  "timeRange": "any",
  "mainContent": true,
  "maxChars": 30000
}

Field	Description
queries	Search queries. Up to 50 per run.
maxResults	Results per query, 1 to 30. Default 5.
scrapePages	Fetch each result as Markdown. Default on.
region	us-en, uk-en, es-es, de-de, fr-fr, mx-es... or wt-wt.
timeRange	any, day, week, month, year.
mainContent	Remove navigation, footers and banners. Default on.
maxChars	Characters per page, 1,000 to 200,000. Default 30,000.

Output example

{
  "query": "best open source CRM for startups",
  "region": "us-en",
  "position": 1,
  "title": "7 Best Open Source CRMs for Startups",
  "url": "https://techsy.io/en/blog/open-source-crm",
  "displayedUrl": "techsy.io/en/blog/open-source-crm",
  "snippet": "Compare the best open source CRMs for startups...",
  "scraped": true,
  "pageTitle": "7 Best Open Source CRMs for Startups in 2026",
  "description": "We compared Twenty, SuiteCRM, EspoCRM...",
  "markdown": "# 7 Best Open Source CRMs for Startups\n\nChoosing a CRM...",
  "wordCount": 3807,
  "author": null,
  "publishedAt": "2026-08-14T00:00:00.000Z",
  "language": "en",
  "truncated": false,
  "likelyJsRendered": false,
  "error": null,
  "isNew": false,
  "timestamp": "2026-10-08T12:00:00.000Z"
}

Pricing

Pay per event, no subscription:

• Result scraped: $0.001 per result with its page converted to Markdown, or per result in search-only mode. 1,000 results cost $1.
• New result: $0.002 per URL that enters the results of a scheduled query.

Results whose page cannot be read are free. Compare with web-browsing Actors for AI at $1.33 per 1,000 pages plus a fee per search.

Use it as an API

curl -X POST "https://api.apify.com/v2/acts/aolmo~rag-web-search/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["latest EU AI Act guidance"], "maxResults": 3, "timeRange": "month"}'

The response is the list of result rows with Markdown, ready for your prompt. The Actor works as a tool for AI agents through the Apify MCP server. To convert URLs you already have, use aolmo/page-to-markdown; to load a whole website, aolmo/website-to-markdown.

FAQ

Which search engine does it use? DuckDuckGo, which aggregates results from several sources. Results can differ slightly from Google's.

Why do some results have an error? Some sites block automated visits or need JavaScript. Those rows keep the title, URL and snippet and are not charged.

How fast is it? Usually 3 to 10 seconds per query with 5 pages.

Can I monitor a topic? Yes: schedule the Actor and new URLs come with isNew: true.

Where is the data stored between runs? In a Key-Value Store of your account named micro-saas-state-rag-web-search, one key per query. Delete it to start fresh.