OpenClaw Web Search Provider

Give OpenClaw agents a repeatable openclaw web search tool without managing a separate search API key. This Actor runs DuckDuckGo-backed queries and exports clean ranked result records for autonomous research, retrieval, and source discovery. Every row includes the original query, title, resolved destination URL, displayed domain, snippet, provider, rank, and fetch provenance.

What this Actor does

The Actor accepts up to 25 independent web queries in one run. For each query it retrieves the first DuckDuckGo organic result page and saves up to 10 unique results. DuckDuckGo redirect links are resolved before output, so downstream agents receive destination URLs rather than tracking links. The default dataset is JSON-ready and can also be downloaded as CSV, Excel, XML, RSS, or JSONL through Apify.

It is intentionally a search-result provider, not a page-content crawler. It does not download the full text of result pages or generate an AI answer. This narrow contract keeps runs fast, predictable, and easy to connect to an OpenClaw research loop.

Who is it for

• OpenClaw operators replacing a paid search-provider integration with a DuckDuckGo-backed workflow.
• Agent builders who need ranked candidates before a separate page-fetch or extraction step.
• Researchers collecting query-to-source evidence with timestamps.
• Automation teams scheduling a stable set of discovery queries.
• Developers prototyping an openclaw web search API through Apify.

Choose a full-page research Actor when you need article bodies, citations extracted from pages, or answer synthesis. Use this Actor when the useful unit is one ranked search result.

Why use this OpenClaw web search provider

• No separate DuckDuckGo key: configure only the Apify run and proxy settings.
• Agent-ready records: one normalized object per organic result.
• Resolved URLs: output points to destination pages.
• Repeatable provenance: query, provider, result page, and retrieval timestamp stay together.
• Multiple queries: gather a compact research set in one run.
• Bounded retries: challenged identities are rotated up to three times, then the run fails clearly.
• Safe output contract: challenge pages are never returned as successful empty results.

Extracted data

Field	Type	Meaning
query	string	Input phrase that produced the row.
provider	string	Always duckduckgo in this version.
rank	integer	One-based position within the query.
title	string	Organic result title.
url	string	Resolved HTTP or HTTPS destination.
displayedDomain	string or null	Domain/path displayed in the result card.
snippet	string or null	Visible summary when DuckDuckGo supplies one.
provenance.searchUrl	string	Reproducible DuckDuckGo HTML query URL.
provenance.page	integer	Result page number; currently 1.
provenance.fetchedAt	string	UTC ISO 8601 retrieval timestamp.

Getting started

1. Open the Actor in Apify Console.
2. Add one or more concrete phrases to Search queries.
3. Choose a maximum from 1 to 10 results per query.
4. Keep us-en or enter another DuckDuckGo region such as uk-en, de-de, or wt-wt.
5. Keep Apify Proxy enabled; direct cloud IPs are commonly challenged.
6. Start the run.
7. Open Ranked web results in the default dataset.
8. Pass the JSON rows to OpenClaw, an LLM, a database, or a follow-up crawler.

A useful first input is:

{
  "queries": ["OpenClaw web search providers"],
  "maxResultsPerQuery": 5,
  "region": "us-en",
  "proxyConfiguration": { "useApifyProxy": true }
}

Input parameters

queries

Required array of 1–25 non-empty strings. Each query may be up to 300 characters. DuckDuckGo operators such as site:docs.openclaw.ai can narrow a source. Every query is processed independently and retained in its output rows.

maxResultsPerQuery

Optional integer from 1–10; default 10. The limit applies independently to each query. For example, three queries with a limit of five can save up to 15 rows.

region

Optional DuckDuckGo region code; default us-en. Use wt-wt when you do not want a country-language preference. A region influences ranking but does not guarantee that every destination is located there.

proxyConfiguration

Apify Proxy is enabled by default because DuckDuckGo often challenges shared direct IPs. The normal route uses datacenter proxy identities. Users whose Apify account permits residential groups may select one explicitly. The Actor does not silently switch to residential traffic. Disabling the proxy can cause a challenge and a failed run.

Output example

This shortened record reflects current Actor behavior:

{
  "query": "OpenClaw web search providers",
  "provider": "duckduckgo",
  "rank": 1,
  "title": "Web search - OpenClaw",
  "url": "https://docs.openclaw.ai/tools/web",
  "displayedDomain": "docs.openclaw.ai/tools/web",
  "snippet": "web_search searches the web with your configured provider...",
  "provenance": {
    "searchUrl": "https://html.duckduckgo.com/html/?q=OpenClaw%20web%20search%20providers",
    "page": 1,
    "fetchedAt": "2026-09-09T20:46:15.120Z"
  }
}

Search results change over time and by region, so titles, snippets, URLs, and ordering are not fixed.

How much does it cost to search the web for OpenClaw agents?

Pricing uses pay per event:

• one small start event per run;
• one result event for each accepted dataset row;
• no charge for challenge pages, duplicates, rejected links, or failed result cards.

At the BRONZE tier, the Actor charges $0.0005 per run plus $0.0014 per saved result. A run saving 5 results costs $0.0075 in Actor event charges. A run saving 10 results costs $0.0145. A multi-query run saving 25 results costs $0.0355. Apify platform usage and proxy consumption may also apply according to your plan. Prices decline on higher Apify tiers; see the live pricing panel for the applicable tier.

OpenClaw research workflow

A practical autonomous workflow is:

1. Let an agent generate several focused search questions.
2. Run this Actor with those questions in queries.
3. Group returned rows by query and preserve rank.
4. Filter destinations by domain policy or snippet relevance.
5. Send selected URLs to a page-fetch or page-research tool.
6. Store provenance.fetchedAt with the resulting evidence.
7. Schedule the same Task later when change tracking is required.

The Actor itself does not compare historical runs. Use an Apify schedule plus your database, dataset diff, or automation platform for monitoring.

Source-specific search

DuckDuckGo search syntax can target a real domain without introducing a separate input mode:

{
  "queries": ["site:docs.openclaw.ai web search configuration"],
  "maxResultsPerQuery": 5,
  "region": "us-en"
}

This is useful for documentation discovery, but it does not guarantee exhaustive indexing of the domain.

API usage with cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~openclaw-web-search-provider/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["OpenClaw DuckDuckGo search setup"],"maxResultsPerQuery":5,"region":"us-en"}'

Keep tokens in secrets or environment variables, never in committed code. Use the asynchronous run endpoint for larger scheduled workflows.

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/openclaw-web-search-provider').call({
  queries: ['OpenClaw web search without Brave'],
  maxResultsPerQuery: 5,
  region: 'us-en',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/openclaw-web-search-provider").call(run_input={
    "queries": ["OpenClaw web search providers"],
    "maxResultsPerQuery": 5,
    "region": "us-en",
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)

MCP setup for Claude Desktop, Cursor, and VS Code

Add the Actor to Claude Code through Apify MCP:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/openclaw-web-search-provider"

Claude Desktop, Cursor, and VS Code can use this equivalent JSON configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/openclaw-web-search-provider"
    }
  }
}

Example prompts:

• “Search for OpenClaw web search providers and return the five highest-ranked source URLs.”
• “Find official OpenClaw documentation about web search configuration.”
• “Run two queries about DuckDuckGo setup and search without Brave, then group evidence by query.”

Scheduling and integrations

Create an Apify Task when the same query set should run repeatedly. Add a schedule for daily, weekly, or monthly execution. Use webhooks to notify an automation after success or failure. Send dataset rows to Make, Zapier, Google Sheets, a vector database, or your own service. When monitoring, compare resolved url values and keep the query as part of the key.

Reliability and retry behavior

DuckDuckGo can return an automated-request challenge instead of results. The Actor recognizes that challenge and rotates to a fresh configured proxy identity. Each query has at most three attempts with a short bounded backoff. Transient network failures, HTTP 429 responses, and temporary 5xx responses use the same bound. After exhaustion, the run fails with the affected query rather than reporting false success. Already saved rows are charged and written only after a complete query response is accepted.

Limits

• Up to 25 queries per run.
• Up to 10 first-page organic results per query.
• Web results only; no news, image, video, ad, or instant-answer mode.
• No full-page content extraction or answer generation.
• No guarantee that DuckDuckGo indexes every relevant page.
• Ranking can vary with region, time, and upstream changes.
• Snippets and displayed domains may be absent.
• Proxy access depends on the user's Apify plan and selected groups.

Legality and responsible use

Search-result pages and destination URLs can contain public personal or copyrighted information. Use the Actor only for lawful purposes and process personal data under an appropriate legal basis. Respect website terms, robots guidance, intellectual-property rights, and applicable privacy law. Do not use search results to make high-impact decisions about people without independent verification. The Actor does not bypass login walls or collect private account data.

Troubleshooting

The run says DuckDuckGo returned a challenge

Keep Apify Proxy enabled and retry later. If your plan permits residential proxy use, select the appropriate residential group explicitly. Do not disable retries or treat the challenge page as an empty result set.

A query returns no rows

Try the same query in DuckDuckGo manually and simplify overly narrow operators. A recognizable upstream “no results” response is valid and remains uncharged for result events.

The run rejects my input

Confirm queries is a non-empty string array, the per-query limit is 1–10, and the region looks like us-en or wt-wt. The Actor rejects blank strings and more than 25 queries.

Why are results different from another provider?

Search engines use different indexes and ranking systems. This Actor reports DuckDuckGo's visible organic ordering; it does not emulate Brave, Google, Tavily, or Bing.

FAQ

Is this an official OpenClaw or DuckDuckGo product?

No. It is an independent Apify Actor that formats public DuckDuckGo web results for OpenClaw-compatible workflows.

Does it need a DuckDuckGo API key?

No separate DuckDuckGo key is required. An Apify account and suitable proxy access are needed for reliable cloud execution.

Can it search more than ten results for one query?

Not in this release. The first-page limit avoids unreliable pagination-token sessions and keeps the output contract honest. Use several focused queries instead of assuming deeper result pages are stable.

Can I use it as an openclaw web search free option?

You can evaluate it under the allowances of your Apify plan. Actor event charges and platform/proxy usage still apply, so it is not advertised as unlimited free search.

Are duplicates removed?

Yes, resolved URLs are deduplicated within each query. The same URL may appear once for each different query because that association is useful evidence.

Does it fetch destination pages?

No. Connect the output URLs to a page extraction Actor when full text is required.

Related Automation Lab Actors

• LLM Web Page Research Browser for fetching selected pages and returning research-ready content.
• Yandex Search Results Scraper when a Yandex-specific result set is required.

These tools are complementary rather than interchangeable: this Actor supplies DuckDuckGo-ranked discovery records.