# Changelog of DuckDuckGo News Scraper (`searchapi/duckduckgo-news-scraper`) Actor

- **URL**: https://apify.com/searchapi/duckduckgo-news-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/searchapi/duckduckgo-news-scraper.md

## Changelog

### \[3.1.0] - 2026-09-22

#### Changed

- Removed the public debug input and debug-only log-level switch.
- Added stable dataset schema requirements, a `fullDetails` contract, and a dataset audit that checks populated fields, positions, timestamps, duplicate IDs, sensitive keys, and schema parity.
- Added a live sample from the public DuckDuckGo News endpoint and refreshed actor metadata.

#### Verified

- A fresh run for `artificial intelligence` returned 20 source-backed records from the current public News endpoint.
- Headed Chrome showed the corresponding current News cards with publisher labels, titles, relative times, snippets, thumbnails, and article links.

### \[3.0.0] - 2026-08-29

#### Changed

- Replaced the broken browser-only response listener with fast API-first extraction from DuckDuckGo's public structured News endpoint.
- Added fair multi-query processing, bounded continuation pagination/concurrency/retries/timeouts, stable query-specific IDs, exact date/sort/region/Safe Search context, and a truthful 39-field schema.
- Switched to the lightweight Node Actor image and deterministic `npm ci` build.

#### Fixed

- Proxy input is now honored; direct access remains the default and `GOOGLE_SERP` is rejected as incompatible.
- Corrected DuckDuckGo country-language metadata and epoch timestamp normalization.
- Status, content type, JSON shape, pagination host/path, and challenge state are validated before use.
- Removed the missing Firefox runtime, browser fingerprint hacks, CAPTCHA guidance, unused browser/error/selector modules, opaque transport-field risk, and three high-severity production dependency findings.

All notable changes to the DuckDuckGo News Scraper are documented here.

### \[2.0.1] - 2026-06-13

#### Verified

- End-to-end run against `duckduckgo.com` (`?iar=news&ia=news`) for query "artificial intelligence" returned **20/20 records** with all fields populated:
  - 20/20 have `title`, `url`, `source`, `excerpt`, `publishedAt` (ISO)
  - 14/20 have `thumbnail` (some news items don't expose an image in the lite API; this is a source limitation, not a scraper bug)
  - All `publishedAt` values parse correctly from the Unix timestamp in the `news.js` API
  - `relativeDate` (e.g. "2 days ago") is preserved alongside the absolute ISO timestamp
- API intercept (`/news.js?o=json`) approach in `main.js` is working as designed.

### \[2.0.0] - 2026-06-12

#### Added

- **Schema expanded** from 25 to **42 fields**.
- **New fields**: `type`, `resultType`, `titleHtml`, `url`, `displayUrl`, `domain`, `favicon`, `excerpt`, `description`, `imageUrl`, `date`, `tags`, `isAd`, `comments`, `query`, `searchUrl`, `market`, `searchMetadata`.
- **New view**: `fullDetails` — all 42 fields.
- **INPUT.json** added at project root for local validation.

### \[2.0.0] - 2026-06-12 (DDG refactor)

#### Added

- **Identity & metadata**: `position`, `page`.
- **News**: `title`, `link`, `source`, `sourceDomain`, `author`, `authorUrl`.
- **Snippet**: `snippet`, `thumbnail`.
- **Dates**: `publishedAt`, `publishedAtRaw`, `relativeTime`.
- **Categorisation & flags**: `category`, `isSponsored`, `isBreaking`.
- **Engagement**: `commentCount`, `shareCount`, `relatedArticlesCount`, `summary`.
- **Run context**: `searchQuery`, `country`, `language`, `scrapedAt`.
- **Shared validator/normalizer** wired to `_canonical/`.

### \[1.0.0] - 2025-05-20

#### Added

- Initial production release of the DuckDuckGo News Scraper.
- `PlaywrightCrawler`-based architecture using Firefox (`apify/actor-node-playwright-firefox`).
- Modular project structure.
- `validate-datasets.js` for local dataset output validation.
- `Dockerfile` using `apify/actor-node-playwright-firefox` image.
- `README.md` with full Apify usage guide.