DuckDuckGo Maps Scraper

Scrapes the public DuckDuckGo Maps results page and returns normalized local business cards. The actor reads the visible Maps sidebar rendered in Firefox, then preserves the values shown there: names, addresses, phones, rating providers, ratings, review counts, categories, prices, hours, open status, review text, thumbnails, and search provenance.

Input

Field	Required	Default	Description
query	Yes	—	Place or business query, such as coffee shops.
location	No	—	City, neighborhood, or address appended as near <location>.
maxItems	No	50	Number of cards to return, from 1 to 500.
category	No	—	Case-insensitive category filter.
minRating	No	—	Keep cards with a known numeric rating at or above the threshold from 0 to 5; unrated cards are excluded.
openNow	No	false	Keep cards whose visible status starts with Open.
region	No	us-en	DuckDuckGo market code.
proxyConfiguration	No	—	Optional Apify proxy settings.
maxRequestRetries	No	2	Retries for transient navigation and server failures, from 0 to 5. A detected CAPTCHA or access denial ends the run without retrying.
requestTimeoutSecs	No	45	Maximum page navigation wait, from 10 to 120 seconds.
headless	No	true	Run the browser without a visible window.

Example:

{
  "query": "coffee shops",
  "location": "New York",
  "maxItems": 20,
  "region": "us-en"
}

The public form prefill and INPUT.json provide a bounded preview: query is artificial intelligence, location is empty, maxItems is 2, and maxRequestRetries is 0. This requests https://duckduckgo.com/?q=artificial+intelligence&ia=maps&iaxm=maps&kl=us-en once. The proxy prefill now explicitly sets useApifyProxy: false, establishing a fresh direct baseline. It does not preserve the older proxy-enabled form prefill. Runtime defaults in the table above remain unchanged, and explicitly supplied proxy settings remain supported. If a requested proxy cannot be configured, the run fails before launching a browser instead of silently making a direct request.

Output

Every record has a stable generated id and uid, output and source positions, the visible business fields, and provenance fields such as sourceUrl, dataSource, retrievalMethod, searchUrl, searchMetadata, and scrapedAt.

The dataset declares more than 70 possible fields. Fields unavailable on the current public card are omitted. The overview view contains the common business fields; fullDetails exposes the complete normalized record.

Local development

npm install
npm run lint
npm test
npm run validate:dataset
npm run audit:dataset
apify run --purge --input-file INPUT.json

The actor pins Playwright 1.62.1 and the matching Apify Playwright Firefox 22-1.62.1 image. The Docker build installs locked dependencies and verifies that the matching Firefox executable exists. Firefox launches with its standard sandbox and TLS behavior, a 30-second launch timeout, and the existing viewport and locale. It does not use a Chrome executable override. This compatibility change avoids a Chromium namespace launch failure without disabling Chromium's sandbox or TLS verification. New runtime records report live_firefox_dom; the historical Chrome sample keeps its original provenance.

.actorignore and .dockerignore exclude local storage, dependencies, caches, credentials, logs, and QA artifacts from uploads and image copies. The test suite uses mocked browser responses and local DOM fixtures and makes no source requests. The place-detail fixture reconstructs the observed heading and Address/Phone/Hours definition list; it is test evidence, not an actor dataset.

Browser termination diagnostics

Firefox is launched directly through Playwright, not through Crawlee's browser launcher. No Crawlee sandbox-disabling defaults are applied. An unexpected page crash, page/context closure, or browser disconnect is recorded with the current operation and elapsed time. Runtime snapshots before launch, after navigation, and on failure record Node memory, the numeric APIFY_MEMORY_MBYTES allocation when available, and readable cgroup v1/v2 memory limits, usage, peak, and OOM counters. Unavailable counters remain unknown. Snapshots exclude page content, headers, proxy credentials, and other environment variables.

A positive oomKillEventsSinceStart means that the container cgroup reported an OOM kill during the scrape; it does not identify which process was killed. A high memory peak alone does not prove OOM. Browser termination fails immediately without another navigation, and errors from verification checks or result waits are no longer swallowed as missing selectors. Cleanup cannot replace the original failure with a secondary close error.

The October 2 diagnostic preview (run bmDpYN04HQIUP2MEf, build 2.1.2) failed after navigation with Firefox unexpectedly closed and zero dataset rows. Its reported peak was 1,023,983,616 bytes (976.5 MiB) against a 1,024 MiB run allocation. This is consistent with memory pressure but is not an OOM diagnosis. The next separately authorized, bounded diagnostic run should use 2,048 MiB while keeping the same URL, zero navigation retries, direct connection, and standard browser security. Increasing allocation is a diagnostic recommendation, not a verified fix or a change to the deployed run settings.

Source behavior

DuckDuckGo can change its Maps markup, return fewer cards, or show no local results for a query. The actor returns only cards it can read from the live public page and does not invent coordinates, websites, or source identifiers when the page does not expose them.

Unrecognized empty pages and non-HTML responses fail instead of being reported as successful empty scrapes. A zero-card result requires an explicit local empty-state message. Verification routes and known challenge-page variants stop navigation retries and extraction.

The September 22 automated run encountered a CAPTCHA. On October 2 at 20:23 UTC, a separate authorized observation of the same public URL in the existing cloud browser rendered a school name, address, phone, and hours without a visible CAPTCHA or authentication gate. The subsequent Firefox preview failed at runtime as described above, so actor availability remains unverified. A successful fresh actor run and real dataset QA are still required before changing working status. Any new CAPTCHA, verification gate, or HTTP access denial ends the actor run without a second navigation or connection change.