# OpenClaw Web Search Provider (`automation-lab/openclaw-web-search-provider`) Actor

Run DuckDuckGo-backed web queries for OpenClaw agents and export ranked titles, resolved URLs, domains, snippets, query, rank, and fetch provenance.

- **URL**: https://apify.com/automation-lab/openclaw-web-search-provider.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.84 / 1,000 result extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## OpenClaw Web Search Provider

Give OpenClaw agents a repeatable **openclaw web search** tool without managing a separate search API key.
This Actor runs DuckDuckGo-backed queries and exports clean ranked result records for autonomous research, retrieval, and source discovery.
Every row includes the original query, title, resolved destination URL, displayed domain, snippet, provider, rank, and fetch provenance.

### What this Actor does

The Actor accepts up to 25 independent web queries in one run.
For each query it retrieves the first DuckDuckGo organic result page and saves up to 10 unique results.
DuckDuckGo redirect links are resolved before output, so downstream agents receive destination URLs rather than tracking links.
The default dataset is JSON-ready and can also be downloaded as CSV, Excel, XML, RSS, or JSONL through Apify.

It is intentionally a search-result provider, not a page-content crawler.
It does not download the full text of result pages or generate an AI answer.
This narrow contract keeps runs fast, predictable, and easy to connect to an OpenClaw research loop.

### Who is it for

- OpenClaw operators replacing a paid search-provider integration with a DuckDuckGo-backed workflow.
- Agent builders who need ranked candidates before a separate page-fetch or extraction step.
- Researchers collecting query-to-source evidence with timestamps.
- Automation teams scheduling a stable set of discovery queries.
- Developers prototyping an openclaw web search API through Apify.

Choose a full-page research Actor when you need article bodies, citations extracted from pages, or answer synthesis.
Use this Actor when the useful unit is one ranked search result.

### Why use this OpenClaw web search provider

- **No separate DuckDuckGo key:** configure only the Apify run and proxy settings.
- **Agent-ready records:** one normalized object per organic result.
- **Resolved URLs:** output points to destination pages.
- **Repeatable provenance:** query, provider, result page, and retrieval timestamp stay together.
- **Multiple queries:** gather a compact research set in one run.
- **Bounded retries:** challenged identities are rotated up to three times, then the run fails clearly.
- **Safe output contract:** challenge pages are never returned as successful empty results.

### Extracted data

| Field | Type | Meaning |
| --- | --- | --- |
| `query` | string | Input phrase that produced the row. |
| `provider` | string | Always `duckduckgo` in this version. |
| `rank` | integer | One-based position within the query. |
| `title` | string | Organic result title. |
| `url` | string | Resolved HTTP or HTTPS destination. |
| `displayedDomain` | string or null | Domain/path displayed in the result card. |
| `snippet` | string or null | Visible summary when DuckDuckGo supplies one. |
| `provenance.searchUrl` | string | Reproducible DuckDuckGo HTML query URL. |
| `provenance.page` | integer | Result page number; currently `1`. |
| `provenance.fetchedAt` | string | UTC ISO 8601 retrieval timestamp. |

### Getting started

1. Open the Actor in Apify Console.
2. Add one or more concrete phrases to **Search queries**.
3. Choose a maximum from 1 to 10 results per query.
4. Keep `us-en` or enter another DuckDuckGo region such as `uk-en`, `de-de`, or `wt-wt`.
5. Keep Apify Proxy enabled; direct cloud IPs are commonly challenged.
6. Start the run.
7. Open **Ranked web results** in the default dataset.
8. Pass the JSON rows to OpenClaw, an LLM, a database, or a follow-up crawler.

A useful first input is:

```json
{
  "queries": ["OpenClaw web search providers"],
  "maxResultsPerQuery": 5,
  "region": "us-en",
  "proxyConfiguration": { "useApifyProxy": true }
}
```

### Input parameters

#### `queries`

Required array of 1–25 non-empty strings.
Each query may be up to 300 characters.
DuckDuckGo operators such as `site:docs.openclaw.ai` can narrow a source.
Every query is processed independently and retained in its output rows.

#### `maxResultsPerQuery`

Optional integer from 1–10; default `10`.
The limit applies independently to each query.
For example, three queries with a limit of five can save up to 15 rows.

#### `region`

Optional DuckDuckGo region code; default `us-en`.
Use `wt-wt` when you do not want a country-language preference.
A region influences ranking but does not guarantee that every destination is located there.

#### `proxyConfiguration`

Apify Proxy is enabled by default because DuckDuckGo often challenges shared direct IPs.
The normal route uses datacenter proxy identities.
Users whose Apify account permits residential groups may select one explicitly.
The Actor does not silently switch to residential traffic.
Disabling the proxy can cause a challenge and a failed run.

### Output example

This shortened record reflects current Actor behavior:

```json
{
  "query": "OpenClaw web search providers",
  "provider": "duckduckgo",
  "rank": 1,
  "title": "Web search - OpenClaw",
  "url": "https://docs.openclaw.ai/tools/web",
  "displayedDomain": "docs.openclaw.ai/tools/web",
  "snippet": "web_search searches the web with your configured provider...",
  "provenance": {
    "searchUrl": "https://html.duckduckgo.com/html/?q=OpenClaw%20web%20search%20providers",
    "page": 1,
    "fetchedAt": "2026-09-09T20:46:15.120Z"
  }
}
```

Search results change over time and by region, so titles, snippets, URLs, and ordering are not fixed.

### How much does it cost to search the web for OpenClaw agents?

Pricing uses pay per event:

- one small `start` event per run;
- one `result` event for each accepted dataset row;
- no charge for challenge pages, duplicates, rejected links, or failed result cards.

At the BRONZE tier, the Actor charges $0.0005 per run plus $0.0014 per saved result.
A run saving 5 results costs $0.0075 in Actor event charges.
A run saving 10 results costs $0.0145.
A multi-query run saving 25 results costs $0.0355.
Apify platform usage and proxy consumption may also apply according to your plan.
Prices decline on higher Apify tiers; see the live pricing panel for the applicable tier.

### OpenClaw research workflow

A practical autonomous workflow is:

1. Let an agent generate several focused search questions.
2. Run this Actor with those questions in `queries`.
3. Group returned rows by `query` and preserve `rank`.
4. Filter destinations by domain policy or snippet relevance.
5. Send selected URLs to a page-fetch or page-research tool.
6. Store `provenance.fetchedAt` with the resulting evidence.
7. Schedule the same Task later when change tracking is required.

The Actor itself does not compare historical runs.
Use an Apify schedule plus your database, dataset diff, or automation platform for monitoring.

### Source-specific search

DuckDuckGo search syntax can target a real domain without introducing a separate input mode:

```json
{
  "queries": ["site:docs.openclaw.ai web search configuration"],
  "maxResultsPerQuery": 5,
  "region": "us-en"
}
```

This is useful for documentation discovery, but it does not guarantee exhaustive indexing of the domain.

### API usage with cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~openclaw-web-search-provider/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["OpenClaw DuckDuckGo search setup"],"maxResultsPerQuery":5,"region":"us-en"}'
```

Keep tokens in secrets or environment variables, never in committed code.
Use the asynchronous run endpoint for larger scheduled workflows.

### API usage with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/openclaw-web-search-provider').call({
  queries: ['OpenClaw web search without Brave'],
  maxResultsPerQuery: 5,
  region: 'us-en',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### API usage with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/openclaw-web-search-provider").call(run_input={
    "queries": ["OpenClaw web search providers"],
    "maxResultsPerQuery": 5,
    "region": "us-en",
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

### MCP setup for Claude Desktop, Cursor, and VS Code

Add the Actor to Claude Code through Apify MCP:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/openclaw-web-search-provider"
```

Claude Desktop, Cursor, and VS Code can use this equivalent JSON configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/openclaw-web-search-provider"
    }
  }
}
```

Example prompts:

- “Search for OpenClaw web search providers and return the five highest-ranked source URLs.”
- “Find official OpenClaw documentation about web search configuration.”
- “Run two queries about DuckDuckGo setup and search without Brave, then group evidence by query.”

### Scheduling and integrations

Create an Apify Task when the same query set should run repeatedly.
Add a schedule for daily, weekly, or monthly execution.
Use webhooks to notify an automation after success or failure.
Send dataset rows to Make, Zapier, Google Sheets, a vector database, or your own service.
When monitoring, compare resolved `url` values and keep the query as part of the key.

### Reliability and retry behavior

DuckDuckGo can return an automated-request challenge instead of results.
The Actor recognizes that challenge and rotates to a fresh configured proxy identity.
Each query has at most three attempts with a short bounded backoff.
Transient network failures, HTTP 429 responses, and temporary 5xx responses use the same bound.
After exhaustion, the run fails with the affected query rather than reporting false success.
Already saved rows are charged and written only after a complete query response is accepted.

### Limits

- Up to 25 queries per run.
- Up to 10 first-page organic results per query.
- Web results only; no news, image, video, ad, or instant-answer mode.
- No full-page content extraction or answer generation.
- No guarantee that DuckDuckGo indexes every relevant page.
- Ranking can vary with region, time, and upstream changes.
- Snippets and displayed domains may be absent.
- Proxy access depends on the user's Apify plan and selected groups.

### Legality and responsible use

Search-result pages and destination URLs can contain public personal or copyrighted information.
Use the Actor only for lawful purposes and process personal data under an appropriate legal basis.
Respect website terms, robots guidance, intellectual-property rights, and applicable privacy law.
Do not use search results to make high-impact decisions about people without independent verification.
The Actor does not bypass login walls or collect private account data.

### Troubleshooting

#### The run says DuckDuckGo returned a challenge

Keep Apify Proxy enabled and retry later.
If your plan permits residential proxy use, select the appropriate residential group explicitly.
Do not disable retries or treat the challenge page as an empty result set.

#### A query returns no rows

Try the same query in DuckDuckGo manually and simplify overly narrow operators.
A recognizable upstream “no results” response is valid and remains uncharged for result events.

#### The run rejects my input

Confirm `queries` is a non-empty string array, the per-query limit is 1–10, and the region looks like `us-en` or `wt-wt`.
The Actor rejects blank strings and more than 25 queries.

#### Why are results different from another provider?

Search engines use different indexes and ranking systems.
This Actor reports DuckDuckGo's visible organic ordering; it does not emulate Brave, Google, Tavily, or Bing.

### FAQ

#### Is this an official OpenClaw or DuckDuckGo product?

No. It is an independent Apify Actor that formats public DuckDuckGo web results for OpenClaw-compatible workflows.

#### Does it need a DuckDuckGo API key?

No separate DuckDuckGo key is required.
An Apify account and suitable proxy access are needed for reliable cloud execution.

#### Can it search more than ten results for one query?

Not in this release.
The first-page limit avoids unreliable pagination-token sessions and keeps the output contract honest.
Use several focused queries instead of assuming deeper result pages are stable.

#### Can I use it as an openclaw web search free option?

You can evaluate it under the allowances of your Apify plan.
Actor event charges and platform/proxy usage still apply, so it is not advertised as unlimited free search.

#### Are duplicates removed?

Yes, resolved URLs are deduplicated within each query.
The same URL may appear once for each different query because that association is useful evidence.

#### Does it fetch destination pages?

No.
Connect the output URLs to a page extraction Actor when full text is required.

### Related Automation Lab Actors

- [LLM Web Page Research Browser](https://apify.com/automation-lab/llm-web-page-research-browser) for fetching selected pages and returning research-ready content.
- [Yandex Search Results Scraper](https://apify.com/automation-lab/yandex-search-results-scraper) when a Yandex-specific result set is required.

These tools are complementary rather than interchangeable: this Actor supplies DuckDuckGo-ranked discovery records.

# Actor input Schema

## `queries` (type: `array`):

One to 25 web search phrases. Each phrase is searched independently and retained in every output record for provenance.

## `maxResultsPerQuery` (type: `integer`):

Maximum number of unique organic web results saved for each query.

## `region` (type: `string`):

DuckDuckGo result region, such as us-en, uk-en, de-de, or wt-wt for no regional preference.

## `proxyConfiguration` (type: `object`):

Apify Proxy is enabled by default because DuckDuckGo commonly challenges cloud datacenter traffic. Residential groups can be selected when your account permits them.

## Actor input object example

```json
{
  "queries": [
    "OpenClaw web search providers",
    "OpenClaw DuckDuckGo search setup"
  ],
  "maxResultsPerQuery": 10,
  "region": "us-en",
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

Query-linked titles, resolved URLs, domains, snippets, ranks, and fetch provenance.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "OpenClaw web search providers",
        "OpenClaw DuckDuckGo search setup"
    ],
    "maxResultsPerQuery": 10,
    "region": "us-en",
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/openclaw-web-search-provider").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "OpenClaw web search providers",
        "OpenClaw DuckDuckGo search setup",
    ],
    "maxResultsPerQuery": 10,
    "region": "us-en",
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/openclaw-web-search-provider").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "OpenClaw web search providers",
    "OpenClaw DuckDuckGo search setup"
  ],
  "maxResultsPerQuery": 10,
  "region": "us-en",
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call automation-lab/openclaw-web-search-provider --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/openclaw-web-search-provider"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8Q9CztoUTkYEnvWDn/builds/uUVE1VbfPrhfyW9zz/openapi.json