# Web Search API for AI Agents - Brave, DuckDuckGo & Bing (`scrapewise/web-search-scraper`) Actor

Search the web without an API key: Brave, DuckDuckGo and Bing results with title, URL, snippet, domain and date, plus the clean main text of the top pages in markdown for LLMs and RAG. US$ 3 per 1,000 result pages.

- **URL**: https://apify.com/scrapewise/web-search-scraper.md
- **Developed by:** [Scrapewise Data](https://apify.com/scrapewise) (community)
- **Categories:** AI, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.85 / 1,000 search results pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Search API for AI Agents

Web Search Scraper runs **web searches without an API key** on **Brave Search, DuckDuckGo and Bing** and returns clean JSON: position, title, URL, domain, site name, snippet and **publication date**. Turn on `scrapeContent` and it also opens the top results and returns their **clean main text in markdown** (no menus, ads or footers), ready for an LLM, an AI agent tool call or a RAG pipeline.

**US$ 1.00 per 1,000 result pages (Brave pages carry up to 20 results). Page text US$ 0.40 per 1,000 pages. No monthly fee, no start fee. Error rows are free.**

### At a glance

- **Price per 1,000 result pages:** US$ 1.00 (down to US$ 0.85 on Gold and above), no run start fee
- **Price per 1,000 pages read (optional):** US$ 0.40, only pages that gave text
- **Engines:** Brave Search (20 results per page), DuckDuckGo, Bing; one or several merged, duplicates removed
- **Fallback:** If the chosen engine is blocked or empty, Brave and DuckDuckGo take over automatically
- **Freshness:** Past day, week, month or year; publication date on each result when the engine has it
- **Page text:** Main text in markdown or plain text, with title, author, date and length
- **Error rows:** Free, with an `errorCode`

### One real row

Query `how to lower cholesterol naturally`, engines Brave + DuckDuckGo, `scrapeContent` on (collected on 2026-09-30, content cut here):

```json
{
  "type": "result",
  "query": "how to lower cholesterol naturally",
  "engine": "brave",
  "position": 1,
  "engineRank": 1,
  "page": 1,
  "title": "Top 5 lifestyle changes to improve your cholesterol - Mayo Clinic",
  "url": "https://www.mayoclinic.org/diseases-conditions/high-blood-cholesterol/in-depth/reduce-cholesterol/art-20045935",
  "domain": "mayoclinic.org",
  "siteName": "Mayo Clinic",
  "snippet": "Soluble fiber helps block some cholesterol from being absorbed into your bloodstream. You can find soluble fiber in foods such as oatmeal, kidney beans, Brussels sprouts, apples and pears...",
  "content": "High cholesterol raises your risk of heart disease and heart attacks. Medicines can help improve your cholesterol. But if you'd rather start with some healthy lifestyle changes...\n\n## 1. Eat heart-healthy foods\n\n...",
  "contentLength": 5143,
  "contentTruncated": false,
  "contentTitle": "Top 5 lifestyle changes to improve your cholesterol",
  "contentAuthor": "Mayo Clinic Staff",
  "contentDate": "2026-01-21",
  "contentError": null
}
```

### What you can do with it

- **Give an AI agent a search tool:** call the Actor with a query (or through the Apify MCP server) and feed the markdown straight into the prompt. One call returns both the links and the text.
- **RAG and research pipelines:** collect the top pages for hundreds of questions, with their text, title, author and date, in one dataset.
- **SEO and rank tracking:** positions, domains and snippets for your keywords on Brave, DuckDuckGo and Bing, by country and language.
- **News and freshness monitoring:** results from the past day or week, with dates.
- **Lead and market research:** which domains show up for a topic, with site names.

### Input

| Field | What it does |
|---|---|
| `queries` | One query per line. Operators like `site:`, quotes and `-word` work as on the engine |
| `engines` | `brave` (default), `duckduckgo`, `bing`; several are merged rank by rank |
| `maxResults` | Default 10, up to 200 per query |
| `fallbackEngines` | Default on: if the chosen engines return nothing, try Brave, then DuckDuckGo |
| `countryCode`, `languageCode` | Region and language, like `us` + `en`, `de` + `de`, `br` + `pt` |
| `timeRange` | `any`, `day`, `week`, `month`, `year` |
| `scrapeContent` | Also open the top results and return their main text |
| `maxContentPages` | How many top results to read per query (default 5) |
| `contentFormat` | `markdown` (default) or `text` |
| `maxContentChars` | Cut long texts here (default 30,000); `contentLength` keeps the full size |

```json
{
  "queries": ["what is retrieval augmented generation", "site:docs.python.org asyncio"],
  "engines": ["brave", "duckduckgo"],
  "maxResults": 10,
  "timeRange": "year",
  "scrapeContent": true,
  "maxContentPages": 5
}
```

### Output fields

`query`, `engine`, `position` (merged rank), `engineRank`, `page`, `title`, `url`, `domain`, `siteName`, `snippet`, `date` (ISO, from Brave and DuckDuckGo when available), `dateText` (as shown, like "2 days ago"), and with `scrapeContent`: `content`, `contentLength`, `contentTruncated`, `contentTitle`, `contentAuthor`, `contentDate`, `contentLanguage`, `finalUrl` (after redirects), `contentError` (why a page gave no text; not charged), `scrapedAt`.

#### Error codes (never charged)

| errorCode | Meaning |
|---|---|
| `NO_RESULTS` | The engines returned no result for this query |
| `BLOCKED` | The engines did not answer after retries and the fallback; run again |
| `INVALID_INPUT` | A value the Actor does not understand |
| `NOT_REACHED`, `UNEXPECTED` | The run timeout arrived, or something unforeseen broke; only that query is lost |

### Pricing

| Event | Per 1,000 |
|---|---|
| Search results page (with at least one result) | US$ 1.00 |
| Page content extracted (only with `scrapeContent`) | US$ 0.40 |

Ten results from Brave are one page: 1,000 searches of 10 results cost US$ 1.00. With two engines each query uses one page per engine. Adding the text of the top 5 pages costs up to US$ 2.00 more per 1,000 queries. Blocked or empty pages, pages without text and error rows are free, and there is no Apify compute bill on top.

### Limitations

- **No Google.** Google now requires JavaScript for every search, and the ways around it cost more than this whole price. Brave Search runs its own index; DuckDuckGo draws largely on Bing's.
- **Bing** sometimes answers anonymous visitors with loosely related results, so it is never used as a fallback; pick it explicitly if you want it.
- Pages that need a login, are built only with JavaScript or refuse automated visits come back with a `contentError` and are not charged. PDFs are skipped.
- Only public search results and public pages. This Actor respects the sites' terms and does not solve captchas.

### FAQ

**Do I need an API key?**
No. Searches run on Brave Search, DuckDuckGo and Bing without an API key, through the Apify proxy.

**What does a run cost?**
US$ 1.00 per 1,000 result pages with at least one result (a Brave page carries up to 20 results), plus US$ 0.40 per 1,000 pages read when `scrapeContent` is on. 1,000 searches of 10 results on Brave cost US$ 1.00. Empty or blocked pages, pages without text and error rows are free, and there is no Apify compute bill on top.

**Does it search Google?**
No. Google now requires JavaScript for every search. Use `engines` to pick `brave` (default), `duckduckgo`, `bing` or several merged.

**How do I get the text of the pages, not just the links?**
Turn on `scrapeContent`. The Actor opens the top `maxContentPages` results per query (default 5) and returns their main text in `content`, as `markdown` or `text` (`contentFormat`), cut at `maxContentChars`.

**Can an AI agent use it as a search tool?**
Yes. Call the Actor with a query, or through the Apify MCP server, and feed the markdown in `content` straight into the prompt.

**How many results can I get per query?**
Up to 200 with `maxResults` (default 10). Use `timeRange` (`day`, `week`, `month`, `year`) for fresh results, and `countryCode` with `languageCode` for a region.

### Changelog

- **2026-10-17:** FAQ in this page.
- **2026-09-30, 0.1:** first release. Brave, DuckDuckGo and Bing, merged results with automatic fallback, country, language and time filters, optional page text in markdown for LLMs, free error rows.

# Actor input Schema

## `queries` (type: `array`):

Search queries, one per line, as you would type them in a search box (for example best running shoes 2026). Operators such as site:, quotes and -word work as on the engine. Each query returns one row per result with position, title, URL, description and date when the engine shows one. Leave empty to run an example query. Also accepts 'searchQueries', 'query', 'keywords' or 'q'.

## `engines` (type: `array`):

One or more of Brave Search (default, up to 20 results per page and dates), DuckDuckGo and Bing. With several engines the results are merged rank by rank and duplicates removed, which gives a broader view than any single engine. Each engine results page with at least one result is charged, so more engines means more pages. Also accepts 'engine' and 'ddg'.

## `maxResults` (type: `integer`):

Maximum results per query (1 to 200, default 10). Brave returns up to 20 results per page and DuckDuckGo and Bing up to 10, so 50 results on Brave reads 3 pages. You pay per results page. Also accepts 'maxResultsPerQuery'.

## `fallbackEngines` (type: `boolean`):

On (default): if the chosen engines return nothing (blocked or empty), the Actor tries Brave and then DuckDuckGo automatically, so a query rarely comes back empty. Only pages with results are charged. Turn off to use only the engines you chose.

## `countryCode` (type: `string`):

Two-letter country code (default us; for example gb, de, br, fr). Sets the region on each engine, so results and their order match what people in that country see.

## `languageCode` (type: `string`):

Two-letter language code for the results (default en; for example de, pt, es). Combine it with the country, for example de and de for German results from Germany.

## `timeRange` (type: `string`):

Only pages the engine dates within this period: Any time (default), Past day, Past week, Past month or Past year. Useful for news and monitoring queries. Results without a date may be left out by the engine.

## `scrapeContent` (type: `boolean`):

Opens the top results of each query and adds their clean main text (no menus, ads or footers), ready for LLM prompts and RAG. Charged per page that yielded text (200+ characters), on top of the results page; pages that fail are free. Off by default.

## `maxContentPages` (type: `integer`):

How many of the top results to open and read for each query (0 to 200, default 5). Only used when page reading is on. Each page read is charged once, so 5 pages for 10 queries is up to 50 page reads.

## `contentFormat` (type: `string`):

Markdown (default) keeps headings, lists and tables, which LLMs read well. Plain text strips all formatting. Only used when page reading is on.

## `maxContentChars` (type: `integer`):

Page texts longer than this many characters (at least 500, default 30,000) are cut here and flagged with contentTruncated; the full length is in contentLength. Lower it to keep rows small for LLM context windows. Same price.

## `proxyConfiguration` (type: `object`):

Apify Proxy, on by default with the datacenter group. A blocked engine is retried and, with the fallback on, replaced by another engine. Keep the default unless you have your own proxy.

## Actor input object example

```json
{
  "queries": [
    "best running shoes 2026"
  ],
  "engines": [
    "brave"
  ],
  "maxResults": 10,
  "fallbackEngines": true,
  "countryCode": "us",
  "languageCode": "en",
  "timeRange": "any",
  "scrapeContent": false,
  "maxContentPages": 5,
  "contentFormat": "markdown",
  "maxContentChars": 30000,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `resultsCsv` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best running shoes 2026"
    ],
    "engines": [
        "brave"
    ],
    "maxResults": 10,
    "countryCode": "us",
    "languageCode": "en",
    "maxContentPages": 5,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapewise/web-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["best running shoes 2026"],
    "engines": ["brave"],
    "maxResults": 10,
    "countryCode": "us",
    "languageCode": "en",
    "maxContentPages": 5,
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("scrapewise/web-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best running shoes 2026"
  ],
  "engines": [
    "brave"
  ],
  "maxResults": 10,
  "countryCode": "us",
  "languageCode": "en",
  "maxContentPages": 5,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call scrapewise/web-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapewise/web-search-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/msLb303zWADwBdHCt/builds/Adj5zYXhSVj1zitgn/openapi.json