RAG Web Search: Google Results as Clean Markdown avatar

RAG Web Search: Google Results as Clean Markdown

Pricing

from $1.40 / 1,000 readable pages

Go to Apify Store
RAG Web Search: Google Results as Clean Markdown

RAG Web Search: Google Results as Clean Markdown

Search Google and get the top result pages as clean Markdown, text or HTML for LLMs, RAG and AI agents. No login or API key. Blocked, login and paywall pages are free status rows.

Pricing

from $1.40 / 1,000 readable pages

Rating

0.0

(0)

Developer

Cliqto Media

Cliqto Media

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

an hour ago

Last modified

Features

Limited permissions

Share

Type a search query and get the top Google result pages as clean Markdown, plain text or HTML, one Dataset row per page. It is a RAG web search tool for LLM apps and AI agents: no login and no API key, and a typical run with 3 pages takes about 20–50 seconds and costs about $0.009.

Start with the prefilled query apify actors and 3 pages. Pages that block robots, ask for a login or sit behind a paywall are not bypassed: they come back as free status rows, and the Actor reads the next Google result instead.

RAG Web Search: a Google query in, the main text of the top pages out as Markdown, text or HTML

Table of contents

What it does

  1. Searches Google for your query in the country you choose.
  2. Takes the organic results in order. Ads and Google's own pages are skipped. URLs are the real page addresses, not Google redirect links; a result whose redirect link Google does not resolve is skipped and not charged.
  3. Opens each page with a normal HTTP request and keeps only the main content: no menus, footers, cookie banners or scripts.
  4. Saves each readable page as one row with the content in one field: markdown, text or html.
  5. Stops when it has the number of readable pages you asked for, or when the time budget ends.

If you put a URL instead of a query, the Actor skips Google and reads only that page.

Good fit

  • Give an LLM or agent fresh web context for a question (retrieval-augmented generation).
  • Collect the readable text of the top results for a topic before chunking and embedding.
  • Read one known URL as Markdown inside the same workflow.

Not a fit / limits

  • No JavaScript rendering. Pages that build their text in the browser (many web apps, YouTube) come back as empty.
  • No login, cookies or paywall bypass. Such pages come back as login_required or paywall.
  • Up to 10 pages per run and English-language Google results. For more queries, start more runs.
  • PDF and other files are not converted (not_html row).

👉 Try it now: open the input form

How it works

  1. It opens Google in a real Chrome browser through an Apify datacenter proxy. Google often shows a CAPTCHA to such visitors: the Actor does not solve it, it closes the browser and tries again with a new IP address, up to 5 times within about 40 seconds.
  2. If all tries are blocked, it asks Google through Apify's Google search proxy (GOOGLE_SERP). If that answer comes back in an unfamiliar layout, the Actor reads it with a simpler fallback reader and, only if that finds nothing, asks the proxy one more time. The price for you is the same on every way.
  3. Google gives result links as its own redirect links. The Actor asks Google where each one leads; links that Google does not resolve are skipped and not charged (linksSkipped in RUN_SUMMARY).
  4. It reads the result pages with a plain HTTP request (a second try goes through a datacenter proxy), keeps the main content and saves one row per page, plus the run report RUN_SUMMARY.

Most of a run is spent on Google: a typical run takes 20–50 seconds.

👉 Try it now: open the input form

Quick start

Input form: search query or URL (box 1), pages to return (box 2), content format (box 3)

  1. Open the Input tab. Keep the prefilled Search query or URL (query) apify actors, or type your own question (box 1).

  2. Keep Pages to return (maxResults) at the prefilled 3, or set up to 10 (box 2).

  3. Keep Content format (outputFormat) at markdown (default), or choose text or html (box 3).

  4. Click Start at the bottom of the form, then open the Output tab. The Results view shows the URL, title, status and word count of each page (box 4); the Markdown view shows the page content.

    Output table of the prefill run: url, title, fetchStatus and wordCount columns (box 4)

Expected result: 3 rows with fetchStatus: "ok" and Markdown content in about 20–50 seconds, plus free status rows for any pages that could not be read.

👉 Try it now: open the input form

Use cases

Buyer jobInput pathOutput entityKey fieldsNext integration
Web context for an LLM answerquery = the user's question, maxResults 3–5one row per pageurl, title, markdownpass markdown to the model with url as the source
Topic corpus for a vector databasequery per topic, maxResults 10, outputFormat textone row per pagetext, wordCount, urlchunk text, keep url as metadata
Read one known pagequery = a single https:// URLone rowmarkdown, httpStatus, fetchStatussame as above

👉 Try it now: open the input form

Input

Input fields

FieldPlain nameTypeRequiredDefault / prefillAllowed valuesEffectExampleRecommendation
querySearch query or URLstringyesprefill apify actors1–500 characters; Google operators work (site:, quotes, -word); or one http(s) URLwhat Google searches, or the single page to readrag chunking strategieswrite it like a normal Google search
maxResultsPages to returnintegernodefault 3, prefill 31–10how many readable pages you get; status rows do not count53–5 for chat answers, 10 for a corpus
outputFormatContent formatstringnomarkdownmarkdown, text, htmlwhich content field is filledtextMarkdown keeps headings and links; text is smallest
countryCodeGoogle countrystringnoustwo letters (gl), e.g. us, gb, dewhich country's results Google shows; language stays Englishgbuse the country of your users
timeoutSecsTime budget (seconds)integerno6010–120the Actor stops reading pages after this time and returns what it has90raise it if you often get DEADLINE
deliveryConnectorSend results tostring (MCP connector)noemptyone of your Apify MCP connectors (Slack, Notion…)after the run a short summary (count, top pages, dataset link) is posted thereyour Notion connectorleave empty if you read the Dataset yourself
deliveryTargetNotion page or channelstringnoemptya Notion page link or a Slack channel IDwhere the summary goes; Notion needs a page linkhttps://www.notion.so/My-research-…the run never fails because of delivery; RUN_SUMMARY.delivery says what happened

Interacting settings

WhenInput AInput BEffective behaviorExampleCost/result effect
Some top pages are blockedmaxResults—the Actor reads further Google results (up to 2 result pages) to fill the numbermaxResults: 3, 3 blocked → reads results 4–6status rows are free
Slow sitesmaxResultstimeoutSecswhen time runs out the run ends as partial with the rows it already hasmaxResults: 10, timeoutSecs: 30 → fewer rowsyou pay only for saved rows
query is a URLquerycountryCodeGoogle is not used; countryCode is ignored; maxResults is 1https://apify.com/actorsone search event + one page event
Platform Max items is set for the runmaxResultsMax items (run options)the run stops at the smaller number; readable and status rows count togethermaxResults: 5, Max items 2 → 2 rows, limitReached: "MAX_ITEMS"you pay for at most Max items pages
Input copied from another search Actorqueries, searchTerms, keywords, urls, startUrls / maxItems, limit, resultsLimitquery / maxResultsread as query / maxResults (first value of a list); when both are set, query and maxResults win and the log says so; unknown fields are listed in RUN_SUMMARY.ignoredInputFields{"searchTerms": ["rag"], "limit": 4}one search per run

Input examples

Answer a question with web context

{
"query": "rag chunking strategies",
"maxResults": 3,
"outputFormat": "markdown"
}

Expected behavior: one Google search, then the top readable pages. Expected output: 3 rows with rank, url, title, markdown. Limit or cost note: 1 search + 3 page events.

Plain text from the UK results

{
"query": "how to make cold brew",
"maxResults": 5,
"outputFormat": "text",
"countryCode": "gb"
}

Expected behavior: Google UK results; blocked pages are skipped and the next results are read. Expected output: up to 5 ok rows with text, plus free status rows for pages that could not be read.

Read one page

{
"query": "https://apify.com/actors",
"outputFormat": "html"
}

Expected behavior: no Google search; the Actor reads this page only. Expected output: 1 row with cleaned html.

👉 Try it now: open the input form

Output

Output fields

FieldMeaningTypeCan be empty?Example / enumSource / notes
queryyour search query (or the URL)stringnoapify actorsinput
rankposition among Google organic results, 1 = topintegerno1Google; ads are not counted
urlreal page addressstringnohttps://apify.com/actorsGoogle redirect and tracking parameters removed
titlepage title, or the Google title if the page was not readstringyesAutomate anything with Apify Actors · Apifypage <title>
descriptionGoogle snippetstringyesActors are web data automations…Google; null in URL mode
markdownmain content as Markdownstringyes# Automate anything…only when outputFormat is markdown; null on status rows
textmain content as plain textstringyesAutomate anything…only when outputFormat is text
htmlcleaned main-content HTMLstringyes<main><h1>…only when outputFormat is html
wordCountwords in the contentintegerno4270 on status rows
fetchStatusresult of reading the pagestringnook, blocked, login_required, paywall, not_html, empty, timeout, erroronly ok rows are charged
httpStatusHTTP status of the pageintegeryes200null when no response
errorwhy there is no contentstringyesHTTP 403null on ok rows
fetchedAtwhen the page was readstring (ISO 8601)no2026-10-15T04:12:15.881Z—

Only one of markdown, text or html is present in a row, the one you chose, so only the matching view (Markdown, Text or HTML) shows content. Content is cut at 200,000 characters.

Output entities and surfaces

SituationDataset/API/KVS/live viewResult shapeUser action
Normal itemDataset, views Results, Markdown, Text, HTMLok rows first, ordered by rankuse the content field
Domain-empty resultDataset empty; RUN_SUMMARY status not_foundno rowschange the query
Partial resultDataset ok rows + status rows; RUN_SUMMARY status partial with partialReason (DEADLINE, CHARGE_LIMIT, CANCELLED, NOT_ENOUGH_READABLE_PAGES)fewer ok rows than askedraise timeoutSecs or your spending limit, or rephrase
Max items reachedDataset rows up to the platform Max items limit; RUN_SUMMARY status complete, limitReached: "MAX_ITEMS"as many rows as Max items allowsraise Max items in run options
Run abortedrows saved before the abort; run ABORTED, RUN_SUMMARY partialReason: "CANCELLED"the pages read so farstart the run again
Technical failurerun FAILED, status message and RUN_SUMMARY status errorno rows; nothing charged, or only search when Google answered but no page could be read in timestart the run again later

RUN_SUMMARY in the default key-value store also has requested, returned, limitReached, ignoredInputFields, delivery (result of the optional summary delivery), serpRoute, serpFallbackParser (true when Google sent an unfamiliar results layout and a simpler reader was used), linksSkipped (Google result links that led nowhere, skipped and not charged), statusCounts and chargedEvents.

Output example

From a real run with the prefilled input (content trimmed):

{
"query": "apify actors",
"rank": 1,
"url": "https://apify.com/actors",
"title": "Automate anything with Apify Actors · Apify",
"description": "Actors are web data automations that power your AI and operations. Run them in the cloud or locally to scrape websites, process data, or connect APIs, all ...",
"markdown": "# Automate anything with Apify Actors\n\nActors are web data automations that power your AI and operations. …",
"wordCount": 427,
"fetchStatus": "ok",
"httpStatus": 200,
"error": null,
"fetchedAt": "2026-10-15T04:12:15.881Z"
}

A status row has the same fields with no content field, for example "fetchStatus": "blocked", "httpStatus": 403, "wordCount": 0.

👉 Try it now: open the input form

Pricing

The Actor uses pay-per-event pricing. You pay only for these events; platform usage is included.

EventWhen it is chargedFree planBronzeSilverGold+
searchonce per run that gets an answer: one Google search (also when Google has no results) or one URL read$0.003$0.00285$0.0027$0.00255
pageper saved row with fetchStatus: "ok"$0.002$0.0018$0.0016$0.0014

search costs more than page because one Google search is the expensive part of a run: a real browser, new IP addresses when Google blocks, and sometimes the Google search proxy. Reading a page is one plain request.

Status rows (blocked, login_required, paywall, not_html, empty, timeout, error) and skipped Google links are free. If Google cannot be reached, the run fails and nothing is charged. If your spending limit does not cover one search event, the run stops before opening Google. If your spending limit is lower than the next row, the run stops as partial and keeps only the paid rows.

Cost examples

Free-plan prices:

ScenarioInput scopeExpected eventsBase chargeOptional chargeStop/cap behavior
TypicalmaxResults 31 search + 3 pages$0.009nonestops at 3 ok rows
One pagemaxResults 1, or one URL1 search + 1 page$0.005none—
LargestmaxResults 10up to 1 search + 10 pagesup to $0.023nonestops at 10 ok rows; tested runs saved 2–5 (see Limits)
No Google resultsany1 search$0.003nonerun ends not_found
All pages blockedany1 search$0.003nonerun ends partial, status rows only
URL could not be readone URL1 search$0.003nonerun ends partial, one status row

👉 Try it now: open the input form

Limits

  • 1–10 readable pages per run; Google results from up to 2 result pages are tried. The second page is read only when the first gave at least one readable page, at least 30 seconds of the time budget are left, and the first page did not come through the Google search proxy.
  • Google hides some result links (see How it works): in tests on the current version a run with maxResults 10 saved 2–5 pages with the default 60-second budget; it saves more when Google answers on the first browser try and most links resolve. Raise timeoutSecs to 90–120 for more pages.
  • One page request waits up to 15 seconds; the whole run follows timeoutSecs (10–120). Reading Google takes up to about 40 seconds (at most half of the time budget) and, when every try is blocked, up to 30 more seconds through the Google search proxy (rarely 15 more for one retry), plus up to 15 seconds to resolve the result links; with timeoutSecs under 60 such a run may stop before Google answers and fail with nothing charged.
  • Page size up to 5 MB; content up to 200,000 characters.
  • If Google sends a results page the Actor cannot read, up to 2 such pages are kept in the run key-value store as SERP_UNREADABLE_* (HTML without scripts) for diagnosis.
  • The platform Max items run option (maxItems in the API) caps all saved rows, readable pages and status rows, and the pages you pay for; the run then ends complete and RUN_SUMMARY shows limitReached: "MAX_ITEMS".
  • 1024 MB memory is the default: Google is read in a real Chrome. Measured runs of up to 10 pages peaked at 650–1020 MB; give 2048 MB if a run stops with a memory error. With more than 1024 MB the time budget is capped at 120 seconds × 1024 MB / memory (60 s at 2048 MB, 30 s at 4096 MB).

👉 Try it now: open the input form

API, export and integrations

  • API: start the Actor with the Apify API and read the Dataset items; each item is one page.
  • Export: the Dataset downloads as JSON, CSV, Excel or HTML from the Output tab.
  • SDK: call it from the Apify Python or JavaScript client and pass the markdown field to your RAG pipeline (for example LangChain or LlamaIndex loaders).

Send results to Slack or Notion

  • Connect Slack or Notion once in Apify Console → MCP connectors, then pick it in Send results to (deliveryConnector).
  • Put a Slack channel ID or a Notion page link in deliveryTarget. After the run a short summary (page count, top pages, Dataset link) is posted there.
  • Delivery never fails the run: RUN_SUMMARY.delivery in the key-value store says whether it was sent and why not.
{ "query": "apify actors", "maxResults": 3, "deliveryConnector": "<your Notion connector>", "deliveryTarget": "https://www.notion.so/My-research-page-…" }

Use with AI agents (MCP)

  • MCP server URL with only this Actor: https://mcp.apify.com?tools=cliqtomedia/rag-web-search (see Apify MCP server).
  • Smallest useful input:
{ "query": "apify actors", "maxResults": 3 }
  • Key output fields: url (source to cite), title, markdown (page content), fetchStatus (ok = usable), rank.
  • A typical call returns 3 pages in about 20–50 seconds and costs about $0.009 on the Free plan.
  • Limits: 10 pages per call, no JavaScript rendering, logins and paywalls are not opened.

👉 Try it now: open the input form

Troubleshooting

Fewer pages than I asked for

Symptom: the run ends as partial with fewer ok rows. Likely reason: many top results blocked the request or needed JavaScript, Google hid some result links (skipped, not charged), or too little time was left for the next Google results page. Check: partialReason and statusCounts in RUN_SUMMARY. Action: for DEADLINE raise timeoutSecs to 90–120; for NOT_ENOUGH_READABLE_PAGES rephrase the query or add -site: for sites that block.

A known site is always blocked

Symptom: the same domain comes back as blocked with HTTP 403. Likely reason: the site refuses automated requests. Pages are read with a plain request and one retry from another IP address; the Actor does not solve CAPTCHAs or bypass anti-bot checks. Action: use other results, or a source that allows automated access.

The run failed

Symptom: run status FAILED, message about Google. Likely reason: Google showed a CAPTCHA on every try and its search proxy did not answer either. Action: start the run again in a few minutes. Nothing was charged.

👉 Try it now: open the input form

FAQ

Do I need a Google API key or a login?

No. The Actor reads public Google results and public pages. It does not open pages that need a login.

What happens when Google finds nothing?

The run succeeds with status not_found, no rows, and one search event ($0.003 on the Free plan).

Is the content complete and up to date?

It is the page as served at fetchedAt, main content only. Text that loads with JavaScript, comments and hidden blocks are not included. Some sites repeat a lead paragraph in their HTML; it then appears twice.

Can I search in other languages?

Google results are requested in English (hl=en); countryCode changes the country. Pages in other languages are returned as they are.

👉 Try it now: open the input form

👉 Try it now: open the input form

Support

For a bug, question or feature request, open the Actor's Issues tab on Apify. Include the run ID, your input (without private data) and the RUN_SUMMARY message. Never send API tokens, cookies or private data.

⭐ Found it useful? Rate this Actor. Something wrong? Report a problem — we fix reported problems.

👉 Try it now: open the input form