# RAG Web Search: Google Results as Clean Markdown (`cliqtomedia/rag-web-search`) Actor

Search Google and get the top result pages as clean Markdown, text or HTML for LLMs, RAG and AI agents. No login or API key. Blocked, login and paywall pages are free status rows.

- **URL**: https://apify.com/cliqtomedia/rag-web-search.md
- **Developed by:** [Cliqto Media](https://apify.com/cliqtomedia) (community)
- **Categories:** AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 75.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.70 / 1,000 readable pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Type a search query and get the top Google result pages as clean Markdown, plain text or HTML, one Dataset row per page. It is a RAG web search tool for LLM apps and AI agents: no login and no API key, and a typical run with 3 pages takes a few seconds and costs about $0.009.

Start with the prefilled query `apify actors` and 3 pages. Pages that block robots, ask for a login or sit behind a paywall are not bypassed: they come back as free status rows, and the Actor reads the next Google result instead.

![RAG Web Search: a Google query in, the main text of the top pages out as Markdown, text or HTML](https://api.apify.com/v2/key-value-stores/jTJRkFTadyV5kTlj7/records/README-HERO-20261015.png)

### Table of contents

- [What it does](#what-it-does)
- [Quick start](#quick-start)
- [Use cases](#use-cases)
- [Input](#input)
- [Output](#output)
- [Pricing and limits](#pricing-and-limits)
- [API, export and integrations](#api-export-and-integrations)
- [Troubleshooting](#troubleshooting)
- [FAQ](#faq)
- [Related Actors](#related-actors)
- [Support](#support)

### What it does

1. Searches [Google](https://www.google.com/) for your query in the country you choose.
2. Takes the organic results in order. Ads and Google's own pages are skipped. URLs are the real page addresses, not Google redirect links.
3. Opens each page with a normal HTTP request and keeps only the main content: no menus, footers, cookie banners or scripts.
4. Saves each readable page as one row with the content in **one** field: `markdown`, `text` or `html`.
5. Stops when it has the number of readable pages you asked for, or when the time budget ends.

If you put a URL instead of a query, the Actor skips Google and reads only that page.

#### Good fit

- Give an LLM or agent fresh web context for a question (retrieval-augmented generation).
- Collect the readable text of the top results for a topic before chunking and embedding.
- Read one known URL as Markdown inside the same workflow.

#### Not a fit / limits

- No JavaScript rendering. Pages that build their text in the browser (many web apps, YouTube) come back as `empty`.
- No login, cookies or paywall bypass. Such pages come back as `login_required` or `paywall`.
- Up to 10 pages per run and English-language Google results. For more queries, start more runs.
- PDF and other files are not converted (`not_html` row).

### Quick start

1. Open the **Input** tab.
2. Keep the prefilled **Search query or URL** (`query`) `apify actors`, or type your own question.
3. Keep **Pages to return** (`maxResults`) at the prefilled 3.
4. Click **Start**.
5. Open the **Output** tab. The **Results** view shows rank, URL, title, word count and status; the **Markdown** view shows the page content.

Expected result: 3 rows with `fetchStatus: "ok"` and Markdown content, usually in under 10 seconds, plus free status rows for any pages that could not be read.

### Use cases

| Buyer job | Input path | Output entity | Key fields | Next integration |
| --- | --- | --- | --- | --- |
| Web context for an LLM answer | `query` = the user's question, `maxResults` 3–5 | one row per page | `url`, `title`, `markdown` | pass `markdown` to the model with `url` as the source |
| Topic corpus for a vector database | `query` per topic, `maxResults` 10, `outputFormat` `text` | one row per page | `text`, `wordCount`, `url` | chunk `text`, keep `url` as metadata |
| Read one known page | `query` = a single `https://` URL | one row | `markdown`, `httpStatus`, `fetchStatus` | same as above |

### Input

#### Input fields

| Field | Plain name | Type | Required | Default / prefill | Allowed values | Effect | Example | Recommendation |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `query` | Search query or URL | string | yes | prefill `apify actors` | 1–500 characters; Google operators work (`site:`, quotes, `-word`); or one `http(s)` URL | what Google searches, or the single page to read | `rag chunking strategies` | write it like a normal Google search |
| `maxResults` | Pages to return | integer | no | default 3, prefill 3 | 1–10 | how many **readable** pages you get; status rows do not count | `5` | 3–5 for chat answers, 10 for a corpus |
| `outputFormat` | Content format | string | no | `markdown` | `markdown`, `text`, `html` | which content field is filled | `text` | Markdown keeps headings and links; text is smallest |
| `countryCode` | Google country | string | no | `us` | two letters (`gl`), e.g. `us`, `gb`, `de` | which country's results Google shows; language stays English | `gb` | use the country of your users |
| `timeoutSecs` | Time budget (seconds) | integer | no | 40 | 10–120 | the Actor stops reading pages after this time and returns what it has | `60` | raise it if you often get `TIME_LIMIT` |

#### Interacting settings

| When | Input A | Input B | Effective behavior | Example | Cost/result effect |
| --- | --- | --- | --- | --- | --- |
| Some top pages are blocked | `maxResults` | — | the Actor reads further Google results (up to 2 result pages) to fill the number | `maxResults: 3`, 3 blocked → reads results 4–6 | status rows are free |
| Slow sites | `maxResults` | `timeoutSecs` | when time runs out the run ends as `partial` with the rows it already has | `maxResults: 10`, `timeoutSecs: 10` → 6 rows | you pay only for saved rows |
| `query` is a URL | `query` | `countryCode` | Google is not used; `countryCode` is ignored; `maxResults` is 1 | `https://apify.com/actors` | one `search` event + one `page` event |

#### Input examples

##### Answer a question with web context

```json
{
  "query": "rag chunking strategies",
  "maxResults": 3,
  "outputFormat": "markdown"
}
```

Expected behavior: one Google search, then the top readable pages.
Expected output: 3 rows with `rank`, `url`, `title`, `markdown`.
Limit or cost note: 1 `search` + 3 `page` events.

##### Plain text from the UK results

```json
{
  "query": "how to make cold brew",
  "maxResults": 5,
  "outputFormat": "text",
  "countryCode": "gb"
}
```

Expected behavior: Google UK results; blocked pages are skipped and the next results are read.
Expected output: up to 5 `ok` rows with `text`, plus free status rows for pages that could not be read.

##### Read one page

```json
{
  "query": "https://apify.com/actors",
  "outputFormat": "html"
}
```

Expected behavior: no Google search; the Actor reads this page only.
Expected output: 1 row with cleaned `html`.

### Output

#### Output fields

| Field | Meaning | Type | Can be empty? | Example / enum | Source / notes |
| --- | --- | --- | --- | --- | --- |
| `query` | your search query (or the URL) | string | no | `apify actors` | input |
| `rank` | position among Google organic results, 1 = top | integer | no | `1` | Google; ads are not counted |
| `url` | real page address | string | no | `https://apify.com/actors` | Google redirect and tracking parameters removed |
| `title` | page title, or the Google title if the page was not read | string | yes | `Automate anything with Apify Actors · Apify` | page `<title>` |
| `description` | Google snippet | string | yes | `Actors are web data automations…` | Google; `null` in URL mode |
| `markdown` | main content as Markdown | string | yes | `# Automate anything…` | only when `outputFormat` is `markdown`; `null` on status rows |
| `text` | main content as plain text | string | yes | `Automate anything…` | only when `outputFormat` is `text` |
| `html` | cleaned main-content HTML | string | yes | `<main><h1>…` | only when `outputFormat` is `html` |
| `wordCount` | words in the content | integer | no | `427` | `0` on status rows |
| `fetchStatus` | result of reading the page | string | no | `ok`, `blocked`, `login_required`, `paywall`, `not_html`, `empty`, `timeout`, `error` | only `ok` rows are charged |
| `httpStatus` | HTTP status of the page | integer | yes | `200` | `null` when no response |
| `error` | why there is no content | string | yes | `HTTP 403` | `null` on `ok` rows |
| `fetchedAt` | when the page was read | string (ISO 8601) | no | `2026-10-15T04:12:15.881Z` | — |

Only one of `markdown`, `text` or `html` is present in a row, the one you chose, so only the matching view (**Markdown**, **Text** or **HTML**) shows content. Content is cut at 200,000 characters.

#### Output entities and surfaces

| Situation | Dataset/API/KVS/live view | Result shape | User action |
| --- | --- | --- | --- |
| Normal item | Dataset, views **Results**, **Markdown**, **Text**, **HTML** | `ok` rows first, ordered by `rank` | use the content field |
| Domain-empty result | Dataset empty; `RUN_SUMMARY` status `not_found` | no rows | change the query |
| Partial result | Dataset `ok` rows + status rows; `RUN_SUMMARY` status `partial` with `partialReason` (`TIME_LIMIT`, `CHARGE_LIMIT`, `NOT_ENOUGH_READABLE_PAGES`) | fewer `ok` rows than asked | raise `timeoutSecs` or your spending limit, or rephrase |
| Technical failure | run `FAILED`, status message and `RUN_SUMMARY` status `error` | no rows, nothing charged | start the run again later |

`RUN_SUMMARY` in the default key-value store also has `requested`, `returned`, `serpRoute`, `statusCounts` and `chargedEvents`.

#### Output example

From a real run with the prefilled input (content trimmed):

```json
{
  "query": "apify actors",
  "rank": 1,
  "url": "https://apify.com/actors",
  "title": "Automate anything with Apify Actors · Apify",
  "description": "Actors are web data automations that power your AI and operations. Run them in the cloud or locally to scrape websites, process data, or connect APIs, all ...",
  "markdown": "# Automate anything with Apify Actors\n\nActors are web data automations that power your AI and operations. …",
  "wordCount": 427,
  "fetchStatus": "ok",
  "httpStatus": 200,
  "error": null,
  "fetchedAt": "2026-10-15T04:12:15.881Z"
}
```

A status row has the same fields with no content field, for example `"fetchStatus": "blocked", "httpStatus": 403, "wordCount": 0`.

### Pricing and limits

#### Pricing unit

The Actor uses pay-per-event pricing. You pay only for these events; platform usage is included.

| Event | When it is charged | Free plan | Bronze | Silver | Gold+ |
| --- | --- | ---: | ---: | ---: | ---: |
| `search` | once per run that gets an answer: one Google search (also when Google has no results) or one URL read | $0.003 | $0.00285 | $0.0027 | $0.00255 |
| `page` | per saved row with `fetchStatus: "ok"` | $0.002 | $0.0019 | $0.0018 | $0.0017 |

Status rows (`blocked`, `login_required`, `paywall`, `not_html`, `empty`, `timeout`, `error`) are free. If Google cannot be reached, the run fails and nothing is charged. If your spending limit is lower than the next row, the run stops as `partial` and keeps only the paid rows.

#### Cost examples

Free-plan prices:

| Scenario | Input scope | Expected events | Base charge | Optional charge | Stop/cap behavior |
| --- | --- | ---: | ---: | ---: | --- |
| Typical | `maxResults` 3 | 1 search + 3 pages | $0.009 | none | stops at 3 `ok` rows |
| One page | `maxResults` 1, or one URL | 1 search + 1 page | $0.005 | none | — |
| Largest | `maxResults` 10 | 1 search + 10 pages | $0.023 | none | stops at 10 `ok` rows |
| No Google results | any | 1 search | $0.003 | none | run ends `not_found` |
| All pages blocked | any | 1 search | $0.003 | none | run ends `partial`, status rows only |
| URL could not be read | one URL | 1 search | $0.003 | none | run ends `partial`, one status row |

#### Limits

- 1–10 readable pages per run; Google results from up to 2 result pages are tried.
- One page request waits up to 15 seconds; the whole run follows `timeoutSecs` (10–120).
- Page size up to 5 MB; content up to 200,000 characters.
- 256 MB memory is enough; measured runs used under 60 MB.

### API, export and integrations

- **API:** start the Actor with the [Apify API](https://docs.apify.com/api/v2) and read the Dataset items; each item is one page.
- **Export:** the Dataset downloads as JSON, CSV, Excel or HTML from the **Output** tab.
- **SDK:** call it from the [Apify Python or JavaScript client](https://docs.apify.com/api/client/python/) and pass the `markdown` field to your RAG pipeline (for example LangChain or LlamaIndex loaders).

#### Use with AI agents (MCP)

- MCP server URL with only this Actor: `https://mcp.apify.com?tools=cliqtomedia/rag-web-search` (see [Apify MCP server](https://docs.apify.com/platform/integrations/mcp)).
- Smallest useful input:

```json
{ "query": "apify actors", "maxResults": 3 }
```

- Key output fields: `url` (source to cite), `title`, `markdown` (page content), `fetchStatus` (`ok` = usable), `rank`.
- A typical call returns 3 pages in about 1–7 seconds and costs about $0.009 on the Free plan.
- Limits: 10 pages per call, no JavaScript rendering, logins and paywalls are not opened.

### Troubleshooting

#### Fewer pages than I asked for

Symptom: the run ends as `partial` with fewer `ok` rows.
Likely reason: many top results blocked the request or needed JavaScript, or the time budget ended.
Check: `partialReason` and `statusCounts` in `RUN_SUMMARY`.
Action: for `TIME_LIMIT` raise `timeoutSecs`; for `NOT_ENOUGH_READABLE_PAGES` rephrase the query or add `-site:` for sites that block.

#### A known site is always `blocked`

Symptom: the same domain comes back as `blocked` with HTTP 403.
Likely reason: the site refuses automated requests. The Actor does not solve CAPTCHAs or bypass anti-bot checks.
Action: use other results, or a source that allows automated access.

#### The run failed

Symptom: run status `FAILED`, message about Google.
Likely reason: Google did not return a readable results page on any route.
Action: start the run again in a few minutes. Nothing was charged.

### FAQ

#### Do I need a Google API key or a login?

No. The Actor reads public Google results and public pages. It does not open pages that need a login.

#### What happens when Google finds nothing?

The run succeeds with status `not_found`, no rows, and one `search` event ($0.003 on the Free plan).

#### Is the content complete and up to date?

It is the page as served at `fetchedAt`, main content only. Text that loads with JavaScript, comments and hidden blocks are not included. Some sites repeat a lead paragraph in their HTML; it then appears twice.

#### Can I search in other languages?

Google results are requested in English (`hl=en`); `countryCode` changes the country. Pages in other languages are returned as they are.

### Related Actors

- [PDF & Image to Text OCR](https://apify.com/cliqtomedia/pdf-text-ocr) — for results that come back as `not_html` PDF files.
- [Website Metadata & Open Graph Scraper](https://apify.com/cliqtomedia/website-metadata-open-graph-scraper) — titles, descriptions and preview images for a list of URLs.

### Support

For a bug, question or feature request, open an issue on the **Issues** tab of this Actor. Include the run ID, your input (without private data) and the `RUN_SUMMARY` message. Never send API tokens or cookies.

# Changelog

This Actor's version history is a separate document: https://apify.com/cliqtomedia/rag-web-search/changelog.md

# Actor input Schema

## `query` (type: `string`):

What to search on Google, e.g. 'rag chunking strategies'. Google operators work (site:, quotes, -word). A single http(s) URL skips the search and reads only that page. Up to 500 characters.

## `maxResults` (type: `integer`):

How many readable pages to return (1–10). Blocked, login, paywall and empty pages do not count: the Actor moves on to the next Google results to fill the number.

## `outputFormat` (type: `string`):

Format of the page content. The Dataset field is named after it: markdown, text or html.

## `countryCode` (type: `string`):

Two-letter country code for Google results (gl), e.g. us, gb, de. Results are in English.

## `timeoutSecs` (type: `integer`):

The Actor stops reading pages after this many seconds and returns what it has (10–120). Slow sites are skipped as timeout rows.

## Actor input object example

```json
{
  "query": "apify actors",
  "maxResults": 3,
  "outputFormat": "markdown",
  "countryCode": "us",
  "timeoutSecs": 40
}
```

# Actor output Schema

## `results` (type: `string`):

Rank, URL, title, status and word count of each page.

## `markdown` (type: `string`):

Clean Markdown of each page (outputFormat markdown).

## `text` (type: `string`):

Plain text of each page (outputFormat text).

## `html` (type: `string`):

Cleaned HTML of each page (outputFormat html).

## `runSummary` (type: `string`):

Status, stop reason, Google route, status counts and charged events.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "apify actors",
    "maxResults": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("cliqtomedia/rag-web-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "apify actors",
    "maxResults": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("cliqtomedia/rag-web-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "apify actors",
  "maxResults": 3
}' |
apify call cliqtomedia/rag-web-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cliqtomedia/rag-web-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gR8aU744Zd3rzSsPY/builds/9and8N4kCcedDl7F8/openapi.json