# DuckDuckGo SERP Scraper — Web, News, Images, Videos & More (`jaybird/duckduckgo-scraper`) Actor Full-SERP DuckDuckGo scraper: web, news, images, videos, autocomplete suggestions and instant answers. HTTP-only, reliability-first, clean JSON for SEO and AI/agent pipelines. - **URL**: https://apify.com/jaybird/duckduckgo-scraper.md - **Developed by:** [Jaybird Technologies](https://apify.com/jaybird) (community) - **Categories:** AI, Agents, SEO tools - **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $0.30 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## DuckDuckGo Scraper Scrape **full DuckDuckGo search results** — web, news, images, videos, autocomplete suggestions, and instant answers — and get clean, structured JSON. A fast, privacy-friendly **DuckDuckGo search API** alternative for SEO research, market analysis, and AI/agent (RAG) pipelines. ### What does DuckDuckGo Scraper do? This Actor extracts search results from [DuckDuckGo](https://duckduckgo.com) across **six content types in one run**: **web**, **news**, **images**, **videos**, **suggestions** (autocomplete), and **instant answers**. It returns clean, consistent JSON with stable field names, result positions, and pagination — ready to download or pipe straight into an app, dashboard, or LLM. It is **HTTP-first** — five of the six types (web, news, videos, suggestions, instant answers) use a fast, lightweight HTTP path with no browser; only **images** use a headless browser (DuckDuckGo blocks non-browser image requests). It is reliability-focused and built to keep working under DuckDuckGo's anti-bot measures via proxy rotation and automatic retries. ### Why use DuckDuckGo Scraper? - 🔎 **Full SERP coverage** — web, news, images, videos, suggestions, and instant answers, not just web links. - 🤖 **AI/agent ready** — flat, predictable JSON with explicit `null`s (never silently missing fields) makes it a drop-in "search tool" for RAG pipelines and AI agents. - 🧭 **SEO & market research** — track rankings, snippets, related questions, and news across regions. - 🌍 **Region, safe-search & time filters** — localize results and restrict by recency or a custom date range. - 🛡️ **Reliability-first** — datacenter proxies by default with automatic **residential fallback** on blocks, retries with backoff, and graceful per-type degradation (one type failing never kills the whole run). - ⚡ **Fast & cheap to run** — an HTTP-first design (browser only for images) keeps compute per result low. - 🗓️ **Apify platform perks** — schedule runs, call via API, integrate with 5,000+ tools, and monitor everything. ### What data can DuckDuckGo Scraper extract? Common fields appear on every item; per-type fields appear based on `type`. | Field | Type | Description | |---|---|---| | `query` | string | The search query that produced the item | | `type` | string | `web`, `news`, `image`, `video`, `suggestion`, or `instant_answer` | | `position` | integer | 1-based rank within the query+type, continuous across pages | | `title` | string | Result title (web, news, images, videos) | | `url` | string | Result URL | | `snippet` | string | Description/snippet (web, news) | | `source` | string | Publisher/source (news) | | `date` / `relativeTime` | string | Publish time (news) | | `imageUrl` / `thumbnailUrl` | string | Media URLs (images, videos, news) | | `duration` / `viewCount` | string/int | Video metadata | | `phrase` | string | Autocomplete suggestion text | | `heading` / `text` / `answer` | string | Instant-answer content | See the **dataset schema** for the full field list per type. ### How to scrape DuckDuckGo 1. Click **Try for free** / open the Actor. 2. Enter one or more **search queries**. 3. Select the **content types** you want (web, news, images, videos, suggestions, instant answers). 4. (Optional) Set **region**, **safe search**, a **time filter** or custom date range, and **max results**. 5. Click **Start** and watch results stream into the dataset. 6. **Download** the data as JSON, CSV, Excel, or HTML — or fetch it via the API. ### How much will it cost to scrape DuckDuckGo? This Actor uses a **pay-per-event** pricing model — you pay only for the results you get, with **no monthly rental**: - **~$0.50 per 1,000 results** for web, news, videos, suggestions, and instant answers (the fast HTTP path). - **~$3.00 per 1,000 image results** — images are billed higher because DuckDuckGo blocks non-browser image requests, so they are fetched via a headless browser (real compute). For example, a run pulling **300 text results + 100 images** costs about **$0.45** (≈ $0.15 + $0.30). The exact, current per-event prices are always shown on the Actor's **Pricing** tab. To minimize cost, request only the content types you need and keep `maxConcurrency` modest. ### Input See the **Input** tab for the full configuration form. Key options: - `queries` — one or more search terms (required). - `contentTypes` — which result types to scrape (defaults to `web`). - `region`, `safeSearch`, `timeFilter` / `dateRangeFrom`+`dateRangeTo`. - `maxResults`, `maxPages`, `maxConcurrency`. Proxying is handled automatically (Apify Proxy datacenter, with residential escalation on blocks) — there is nothing to configure. Example input: ```json { "queries": ["apify web scraping", "duckduckgo api"], "contentTypes": ["web", "news", "suggestions"], "region": "us-en", "safeSearch": "moderate", "maxResults": 50 } ```` ### Output You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. Example items: ```json [ { "query": "duckduckgo api", "type": "web", "position": 1, "title": "DuckDuckGo Instant Answer API", "url": "https://duckduckgo.com/api", "displayUrl": "duckduckgo.com/api", "snippet": "Free Instant Answer API for DuckDuckGo...", "isAd": false, "region": "us-en", "scrapedAt": "2026-06-03T12:00:00.000Z", "schemaVersion": "1" }, { "query": "duckduckgo api", "type": "suggestion", "position": 1, "phrase": "duckduckgo api python", "region": "us-en", "scrapedAt": "2026-06-03T12:00:00.000Z", "schemaVersion": "1" } ] ``` A `RUN_METRICS` record is also saved to the key-value store with reliability stats (block rate, proxy escalations, per-type degradation) and cost-estimation inputs. ### Tips & advanced options - **Reduce cost / blocks:** request only the content types you need and keep `maxConcurrency` low. Proxying is automatic — datacenter first, with residential used only to recover blocked requests. - **Recency:** use `timeFilter` (`day`/`week`/`month`/`year`) or a custom `dateRangeFrom`/`dateRangeTo` range (web, news, videos). - **Localization:** set `region` to a DDG locale like `us-en`, `uk-en`, `de-de`, or `wt-wt` for worldwide. - **Partial results are normal:** if one content type is blocked, the others still return — check `RUN_METRICS` for what degraded. ### Integrations Plug DuckDuckGo Scraper into your stack with no extra code: - **Export** results to **Google Sheets**, **Google Drive**, Amazon S3, or a database. - **Automate** with **Zapier**, **Make**, and **webhooks** that fire when a run finishes. - **Schedule** recurring runs to track rankings, news, or image trends over time. - **Call it from anywhere** via the **Apify API** and the JavaScript / Python SDKs — a clean, drop-in DuckDuckGo search endpoint for your apps and AI agents. ### FAQ, disclaimers, and support **Are ads and knowledge-graph panels included?** DuckDuckGo serves ads and knowledge/instant-answer panels **inconsistently**. This Actor extracts them on a **best-effort basis** when present (`isAd`, instant answers) and does not guarantee them for every query. **How are images scraped?** DuckDuckGo blocks programmatic (non-browser) image requests, so images are fetched through a headless browser that loads the image results page like a real user. This is slower and more compute-intensive than the other types, which is why image results are billed at a higher rate. All other content types use a fast, lightweight HTTP path (no browser). **Does DuckDuckGo have an official search API?** DuckDuckGo offers only a limited Instant Answer API — there is no official API for full web, news, image, or video results. This Actor fills that gap, returning the full SERP as clean JSON, so it works as a practical **DuckDuckGo search API** alternative. **Can I use the results for AI, LLMs, or RAG?** Yes. The output is flat, predictable JSON with stable field names and explicit `null`s, so it drops straight into retrieval-augmented generation (RAG) pipelines and agent "search tools" without custom parsing. **Is it legal to scrape DuckDuckGo?** Scraping publicly available search results is generally legal, and this Actor only collects public data. You are responsible for how you use the results — see the note below on personal data and GDPR. > Our Actors are ethical and do not extract any private user data, such as email addresses, gender, or location. They only extract what the user has chosen to share publicly. We therefore believe that our Actors, when used for ethical purposes by Apify users, are safe. However, you should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. Found a bug or have a feature request? Open an issue on the **Issues** tab. For programmatic access, see the **API** tab. # Actor input Schema ## `queries` (type: `array`): One or more search terms. Each query is run against every selected content type. ## `contentTypes` (type: `array`): Which DuckDuckGo result types to scrape. Note: images are fetched via a headless browser (DuckDuckGo blocks non-browser image requests) and are billed at a higher per-result rate; image runs are slower than the other types. ## `region` (type: `string`): DuckDuckGo locale code, e.g. us-en, uk-en, de-de. Use wt-wt for no region (worldwide). ## `safeSearch` (type: `string`): Content filtering level. ## `timeFilter` (type: `string`): Restrict results to a recent period (applies to web, news, videos). Ignored if a custom date range is set. ## `dateRangeFrom` (type: `string`): Optional start date (YYYY-MM-DD). If set with the end date, overrides the time filter. ## `dateRangeTo` (type: `string`): Optional end date (YYYY-MM-DD). ## `maxResults` (type: `integer`): Maximum results per query per content type. 0 = unlimited. ## `maxPages` (type: `integer`): Maximum result pages to fetch per query per content type. 0 = unlimited (bounded by max results). ## `maxConcurrency` (type: `integer`): How many scraping tasks run in parallel. Lower values reduce the chance of being rate-limited. ## Actor input object example ```json { "queries": [ "apify web scraping" ], "contentTypes": [ "web" ], "region": "wt-wt", "safeSearch": "moderate", "timeFilter": "any", "maxResults": 100, "maxPages": 0, "maxConcurrency": 4 } ``` # Actor output Schema ## `dataset` (type: `string`): No description ## `runMetrics` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "apify web scraping" ] }; // Run the Actor and wait for it to finish const run = await client.actor("jaybird/duckduckgo-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["apify web scraping"] } # Run the Actor and wait for it to finish run = client.actor("jaybird/duckduckgo-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "apify web scraping" ] }' | apify call jaybird/duckduckgo-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jaybird/duckduckgo-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "DuckDuckGo SERP Scraper — Web, News, Images, Videos & More", "description": "Full-SERP DuckDuckGo scraper: web, news, images, videos, autocomplete suggestions and instant answers. HTTP-only, reliability-first, clean JSON for SEO and AI/agent pipelines.", "version": "0.1", "x-build-id": "dbLix9eAKXWLHuchZ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jaybird~duckduckgo-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jaybird-duckduckgo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jaybird~duckduckgo-scraper/runs": { "post": { "operationId": "runs-sync-jaybird-duckduckgo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jaybird~duckduckgo-scraper/run-sync": { "post": { "operationId": "run-sync-jaybird-duckduckgo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "queries": { "title": "Search queries", "minItems": 1, "type": "array", "description": "One or more search terms. Each query is run against every selected content type.", "default": [ "apify" ], "items": { "type": "string" } }, "contentTypes": { "title": "Content types", "type": "array", "description": "Which DuckDuckGo result types to scrape. Note: images are fetched via a headless browser (DuckDuckGo blocks non-browser image requests) and are billed at a higher per-result rate; image runs are slower than the other types.", "items": { "type": "string", "enum": [ "web", "news", "images", "videos", "suggestions", "instantAnswer" ], "enumTitles": [ "Web results", "News", "Images", "Videos", "Suggestions (autocomplete)", "Instant answers" ] }, "default": [ "web" ] }, "region": { "title": "Region / localization", "type": "string", "description": "DuckDuckGo locale code, e.g. us-en, uk-en, de-de. Use wt-wt for no region (worldwide).", "default": "wt-wt" }, "safeSearch": { "title": "Safe search", "enum": [ "off", "moderate", "strict" ], "type": "string", "description": "Content filtering level.", "default": "moderate" }, "timeFilter": { "title": "Time filter", "enum": [ "any", "day", "week", "month", "year" ], "type": "string", "description": "Restrict results to a recent period (applies to web, news, videos). Ignored if a custom date range is set.", "default": "any" }, "dateRangeFrom": { "title": "Custom date range — from", "type": "string", "description": "Optional start date (YYYY-MM-DD). If set with the end date, overrides the time filter." }, "dateRangeTo": { "title": "Custom date range — to", "type": "string", "description": "Optional end date (YYYY-MM-DD)." }, "maxResults": { "title": "Max results per query/type", "minimum": 0, "type": "integer", "description": "Maximum results per query per content type. 0 = unlimited.", "default": 100 }, "maxPages": { "title": "Max pages per query/type", "minimum": 0, "type": "integer", "description": "Maximum result pages to fetch per query per content type. 0 = unlimited (bounded by max results).", "default": 0 }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "type": "integer", "description": "How many scraping tasks run in parallel. Lower values reduce the chance of being rate-limited.", "default": 4 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```