# RAG Web Browser Lite — Cheap Markdown for AI Agents (`subimpact/rag-web-browser-lite`) Actor

Web search and fetch tool for AI agents and RAG pipelines. Queries Google Search, scrapes the top N pages over raw HTTP, and returns clean Markdown. Fixed $0.002 per page — no surprise compute bills.

- **URL**: https://apify.com/subimpact/rag-web-browser-lite.md
- **Developed by:** [subimpact](https://apify.com/subimpact) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 page extracteds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## RAG Web Browser Lite — Cheap Markdown for AI Agents

A lean, drop-in alternative to `apify/rag-web-browser` for AI agents and RAG pipelines.

**Same output shape, fixed price.** Queries Google Search, scrapes the top N pages, and returns clean Markdown — but over **raw HTTP only** (no Playwright browser), so each page costs a flat **$0.002** instead of variable compute bills.

### Why it's cheaper

| | apify/rag-web-browser | rag-web-browser-lite |
|---|---|---|
| Engine | Crawlee + Playwright + MCP SDK | raw HTTP + cheerio + turndown |
| Browser launch per page | Yes (heavy) | Never |
| Billing | Pay per usage (variable) | **PPE: $0.002/page, fixed** |
| Container | ~1GB+ (Playwright) | ~200MB |
| Output shape | `{crawl, searchResult, metadata, markdown}` | Identical |

The official actor's own docs say raw HTTP is "about two times faster" than the browser tool — we made that the *only* mode.

### Usage

#### Normal mode (one request per run)

```bash
curl -X POST "https://api.apify.com/v2/acts/subimpact~rag-web-browser-lite/runs" \
  -H "Content-Type: application/json" \
  -d '{"query": "best crm for startups", "maxResults": 3}'
```

#### Standby mode (agent-facing HTTP endpoint)

```bash
curl "https://subimpact--rag-web-browser-lite.apify.actor/search?query=apify+mcp&maxResults=2&token=<APIFY_API_TOKEN>"
```

#### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `query` | string | — | Search term or URL (required) |
| `maxResults` | int | 3 | Top N organic results to extract (1-100) |
| `outputFormats` | array | `["markdown"]` | `markdown`, `text`, `html` |
| `gl` / `hl` | string | `us` / `en` | Google country / language |
| `serpProxyGroup` | string | `GOOGLE_SERP` | Proxy group for SERP fetch |
| `serpMaxRetries` | int | 2 | SERP retries (0-5) |
| `proxyConfiguration` | object | Apify proxy | Proxy for target pages |
| `removeElementsCssSelector` | string | nav/footer/script/... | Elements to strip before conversion |

#### Output row

```json
{
  "crawl": { "httpStatusCode": 200, "httpStatusMessage": "OK", "loadedAt": "...", "uniqueKey": "...", "requestStatus": "handled" },
  "searchResult": { "title": "...", "description": "...", "url": "...", "resultType": "ORGANIC", "position": 1 },
  "metadata": { "url": "...", "title": "...", "description": "...", "languageCode": "en", "canonicalUrl": "...", "openGraph": [...], "jsonLd": [...], "headers": {...} },
  "markdown": "# ..."
}
```

### Pricing

- **$0.002 per page extracted** (pay-per-event, `page-extracted`)
- Platform usage (proxy, compute) is paid by the buyer
- **No charge for failed/blocked/empty pages** — only pages with ≥100 chars of real content are billed
- SERP fetch is billed as platform usage (GOOGLE\_SERP proxy), not as an event

### Development

```bash
npm install
npm test        # unit tests (parser + content pipeline)
apify push      # build to Apify
```

### Notes

- SERP parsing reuses the battle-tested goto-map + organic parser from `google-serp-ai-overviews` (verified 2026-08-27: top-organic 100% mappable).
- GOOGLE\_SERP proxy requires `http://` (not `https://`) for the SERP fetch.
- Standby detection: `Actor.getEnv().metaOrigin === 'STANDBY'`; port from `Actor.config.get('containerPort')`.
- Reddit and other JS-heavy sites that block raw HTTP return stub pages — those are not charged (content gate).

# Actor input Schema

## `query` (type: `string`):

Enter Google Search keywords or a URL of a specific web page. Examples:

- <code>san francisco weather</code>
- <code>https://www.cnn.com</code>
- <code>function calling site:openai.com</code>

## `maxResults` (type: `integer`):

The maximum number of top organic Google Search results whose web pages will be extracted. If `query` is a URL, this field is ignored and the Actor only fetches the specific web page.

## `outputFormats` (type: `array`):

Select one or more formats to which the target web pages will be extracted and saved in the resulting dataset.

## `gl` (type: `string`):

Two-letter country code for Google Search, e.g. us, my, uk, sg.

## `hl` (type: `string`):

Two-letter language code for Google Search, e.g. en, ms, zh.

## `serpProxyGroup` (type: `string`):

Apify Proxy group used for fetching Google Search results.

## `serpMaxRetries` (type: `integer`):

The maximum number of times the Actor will retry fetching the Google Search results on error.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration used for scraping the target web pages.

## `removeElementsCssSelector` (type: `string`):

A CSS selector matching HTML elements that will be removed from the DOM before converting to text, Markdown, or saving as HTML. Set to a non-existent selector like `dummy_keep_everything` to disable removal.

## `requestTimeoutSecs` (type: `integer`):

The maximum time in seconds available for the request, including querying Google Search and scraping the target web pages.

## Actor input object example

```json
{
  "query": "web browser for RAG pipelines -site:reddit.com",
  "maxResults": 3,
  "outputFormats": [
    "markdown"
  ],
  "gl": "us",
  "hl": "en",
  "serpProxyGroup": "GOOGLE_SERP",
  "serpMaxRetries": 2,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "removeElementsCssSelector": "nav, footer, script, style, noscript, svg, img[src^='data:'],\n[role=\"alert\"],\n[role=\"banner\"],\n[role=\"dialog\"],\n[role=\"alertdialog\"],\n[role=\"region\"][aria-label*=\"skip\" i],\n[aria-modal=\"true\"]",
  "requestTimeoutSecs": 40
}
```

# Actor output Schema

## `crawlResults` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "web browser for RAG pipelines -site:reddit.com",
    "proxyConfiguration": {
        "useApifyProxy": true
    },
    "removeElementsCssSelector": `nav, footer, script, style, noscript, svg, img[src^='data:'],
[role="alert"],
[role="banner"],
[role="dialog"],
[role="alertdialog"],
[role="region"][aria-label*="skip" i],
[aria-modal="true"]`
};

// Run the Actor and wait for it to finish
const run = await client.actor("subimpact/rag-web-browser-lite").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "web browser for RAG pipelines -site:reddit.com",
    "proxyConfiguration": { "useApifyProxy": True },
    "removeElementsCssSelector": """nav, footer, script, style, noscript, svg, img[src^='data:'],
[role=\"alert\"],
[role=\"banner\"],
[role=\"dialog\"],
[role=\"alertdialog\"],
[role=\"region\"][aria-label*=\"skip\" i],
[aria-modal=\"true\"]""",
}

# Run the Actor and wait for it to finish
run = client.actor("subimpact/rag-web-browser-lite").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "web browser for RAG pipelines -site:reddit.com",
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "removeElementsCssSelector": "nav, footer, script, style, noscript, svg, img[src^='\''data:'\''],\\n[role=\\"alert\\"],\\n[role=\\"banner\\"],\\n[role=\\"dialog\\"],\\n[role=\\"alertdialog\\"],\\n[role=\\"region\\"][aria-label*=\\"skip\\" i],\\n[aria-modal=\\"true\\"]"
}' |
apify call subimpact/rag-web-browser-lite --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,subimpact/rag-web-browser-lite"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/elSU1GrqEIKfpEnP1/builds/i60bRV2rQyewsf99C/openapi.json