DuckDuckGo Search Scraper

Apify Actor for extracting DuckDuckGo search results for AI research, SEO monitoring, source discovery, and contact discovery workflows.

This actor is the first pilot of the content-intelligence bundle. It intentionally starts with a lightweight HTTP implementation because DuckDuckGo has lower proxy/Cloudflare risk than many ecommerce and social targets in the wider Apify portfolio.

Features

• Multiple search modes: Default (web), images, news, videos
• Batch queries: Run many search queries in one run via a list (one per line; empty rows are removed automatically)
• Single or list: Use one search query or an array; comma-separated or newline-separated text is supported
• Query options: Remove duplicate queries, optional cap on number of queries (max queries)
• Auto pagination: Follow “More results” to collect up to maxResults across pages
• Email discovery: Optional extraction of email addresses from result titles and snippets, with optional domain filter and query suffix for contact/press/legal discovery
• Structured output: Each result includes query, source input, title, URL, canonical URL, snippet, position/rank, mode, listing URL, page, and optional emails
• Proxy support: Optional Apify Proxy/custom proxy configuration for cloud runs

Search Modes

Mode	Description
Default	Standard web search
Images	Image search results
News	News articles
Videos	Video search results

Input

Parameter	Required	Description
searchQuery	Yes*	Single search term or phrase. Used when Search Queries (array) is empty. Comma-separated values are split into multiple queries.
searchQueries	No	List of search terms (e.g. from Bulk edit, one per line). Overrides searchQuery when non-empty. Empty rows are removed.
removeDuplicateQueries	No	Remove duplicate entries in the query list (default: true).
maxQueries	No	Max number of queries to run from the list; 0 = no limit (default: 0).
searchMode	Yes	default, images, news, or videos.
maxResults	No	Max results to extract per run (default: 100, max: 1000).
maxPages	No	Max result pages to crawl (default: 5, max: 100).
autoPagination	No	Click “More results” to get more pages (default: true).
failOnNoResults	No	Fail the run when no dataset items are extracted. Useful for monitoring and paid workflows (default: false).
extractEmails	No	Parse titles and snippets for email addresses and add an emails array to each result (default: false).
emailDomainFilter	No	When extracting emails, keep only addresses containing this domain (e.g. 20minutes.fr). Empty = all emails.
emailSearchSuffix	No	Suffix appended to every search query (e.g. contact email, "nous contacter").

* Either searchQuery or searchQueries must provide at least one query; searchQuery has a default.

Output

Results are stored in the default dataset. Each item looks like:

{
  "source": "duckduckgo",
  "query": "apify actors",
  "sourceInput": "apify actors",
  "searchMode": "default",
  "title": "Result title",
  "url": "https://example.com/page",
  "canonicalUrl": "https://example.com/page",
  "snippet": "Description or snippet from the result",
  "position": 1,
  "rank": 1,
  "page": 1,
  "listingUrl": "https://duckduckgo.com/html/?q=apify+actors",
  "scrapedAt": "2026-06-25T14:48:49.208Z"
}

When Extract emails from results is enabled, each item can include:

{
  "title": "...",
  "url": "...",
  "snippet": "...",
  "position": 1,
  "rank": 1,
  "searchMode": "default",
  "page": 1,
  "emails": ["contact@example.com", "press@example.com"]
}

• position / rank: Rank in the query result list (1-based).
• canonicalUrl: URL cleaned from common tracking parameters.
• listingUrl: DuckDuckGo result page URL used for extraction.
• page: Page number where the result was found.
• searchMode: default, images, news, or videos.
• emails: Present only when email extraction is on; deduplicated per result; filtered by emailDomainFilter if set.

Usage Examples

Single search

{
  "searchQuery": "artificial intelligence",
  "searchMode": "default",
  "maxResults": 50
}

Multiple queries (array)

{
  "searchQueries": [
    "site:example.com contact",
    "site:example.com \"nous contacter\" email"
  ],
  "searchMode": "default",
  "maxResults": 100
}

Email discovery (contact/press)

{
  "searchQueries": [
    "site:20minutes.fr contact email",
    "site:20minutes.fr \"mentions légales\" email",
    "site:20minutes.fr \"presse\" email"
  ],
  "searchMode": "default",
  "extractEmails": true,
  "emailDomainFilter": "20minutes.fr",
  "emailSearchSuffix": " email",
  "maxResults": 100
}

Limit number of queries (test run)

{
  "searchQueries": ["query1", "query2", "query3", "..."],
  "searchMode": "default",
  "maxQueries": 5,
  "maxResults": 20
}

One-result validation run

{
  "searchQuery": "apify web scraping",
  "searchMode": "default",
  "maxResults": 1,
  "maxPages": 1,
  "autoPagination": false,
  "failOnNoResults": true,
  "extractEmails": true
}

For email discovery, emailSearchSuffix is appended to every query. Leave it empty for broad smoke tests, then add suffixes such as contact email or "mentions legales" email for targeted discovery.

Multi-page with pagination

{
  "searchQuery": "web development tutorials",
  "searchMode": "default",
  "maxResults": 100,
  "maxPages": 5,
  "autoPagination": true
}

Image search

{
  "searchQuery": "nature photography",
  "searchMode": "images",
  "maxResults": 30
}

Legal

This Actor is for educational and research use. You are responsible for:

• Complying with DuckDuckGo’s terms of service
• Respecting robots.txt and site terms
• Following applicable laws
• Using the tool in a responsible and ethical way

The developers are not responsible for misuse or violations of terms of service.

Development

Install dependencies:

$npm install --package-lock=false

Run tests:

$npm test

Run a local smoke:

$apify run --purge --input '{"searchQuery":"apify actors","searchMode":"default","maxResults":1,"maxPages":1,"autoPagination":false,"failOnNoResults":true}'