# Changelog of Brave News Scraper (`searchapi/brave-news-scraper`) Actor

- **URL**: https://apify.com/searchapi/brave-news-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/searchapi/brave-news-scraper.md

## Changelog

All notable changes to the Brave News Scraper are documented here.

### \[3.1.0] - 2026-08-29

#### Added

- Safe, non-executing AST parsing of Brave's SvelteKit hydration response with strict query/payload validation and current DOM fallback.
- Single-query, deduplicated multi-query, fair batch, exact one-item, time-filtered, and structural no-results modes.
- Verified `day`, `week`, `month`, and `year` website time filters using Brave's current `tf` values.
- Canonical article URLs, stable SHA-256 IDs, global/query/request positions, source profiles, live-story status, explicit provenance, and all-or-nothing batch writes.
- Bounded concurrent pages, coherent locale/timezone/viewport settings, stable proxy sessions, verified resource blocking, and eleven focused tests.

#### Changed

- Corrected the prior conclusion that Brave News had no structured response or time filter: both are present in the current SvelteKit document and live query behavior.
- Aligned local and Docker Playwright Firefox to `1.62.1`, pinned production dependencies, and removed all vulnerability findings.
- Replaced the incomplete flat output contract with a wrapper-valid 66-field schema and recursive empty-value omission.
- Reduced temporary retries to one by default and never retries explicit challenge pages.

#### Removed

- Fabricated `false` advertising, sponsorship, and breaking-news flags when Brave did not expose those values.
- Opaque hydration debugging, tracking/click, and request state from output.

### \[3.0.0] - 2026-07-20

#### Breaking

- Removed the obsolete `timeRange` and `sortBy` inputs because Brave's current News UI does not expose or honor those filters.
- Limited `maxItems` to 100, matching the current single-page result surface instead of claiming unsupported pagination.

#### Added

- Extracts 51 normalized fields from current Brave News cards, including publisher domain and URL, favicon, plain and HTML snippets, image metadata, Safe Search, engine metadata, and one shared scrape timestamp.
- Supports and verifies Brave's `strict`, `moderate`, and `off` Safe Search preferences.
- Adds bounded response retries, navigation timeout controls, browser headless control, and Apify/custom proxy support.
- Adds automated coverage for input validation, URL construction, relative dates, rich normalization, and URL-based deduplication.

#### Changed

- Replaced the non-functional news API interception path with stable DOM extraction because current Brave News responses contain no structured results API.
- Validates response status and content type, retained query/country/language values, block pages, and Safe Search state before extracting.
- Uses current Brave News card selectors and only emits values backed by the listing; unavailable optional values remain null or empty arrays.
- Updated Apify SDK, Crawlee, Playwright Firefox, Actor schema, Docker image, and dependencies.

### \[2.0.0] - 2026-06-12

#### Added

- **Schema expanded** from 24 to **44 fields**.
- **New fields**: `type`, `resultType`, `titleHtml`, `url`, `displayUrl`, `domain`, `favicon`, `description`, `imageUrl`, `date`, `tags`, `isAd`, `comments`, `query`, `searchUrl`, `market`, `searchMetadata`.
- **New view**: `fullDetails` — all 44 fields in a single table.
- **INPUT.json** added at project root for local validation.

#### Changed

- `dataset_schema.json` now exports 2 views: `overview` and `fullDetails`.

### \[2.0.0] - 2026-06-12 (Brave refactor)

#### BREAKING

- Renamed `url` → `link` (canonical naming, per `_canonical/CONTRIBUTING.md`).
- Replaced ad-hoc dataset shape with the canonical news-vertical schema. All records now pass through `_canonical/normalize.js` `normalizeNewsRecord` before being pushed. The output is a strict subset of `_canonical/schemas/news.schema.json`.

#### Added

- **Identity & metadata**: `page`, `sourceDomain` (derived from `link`).
- **Author**: `author`, `authorUrl`.
- **Dates**: `publishedAt`, `publishedAtRaw`, `relativeTime`.
- **Categorisation & flags**: `category`, `isSponsored`, `isBreaking`.
- **Engagement**: `commentCount`, `shareCount`, `relatedArticlesCount`.
- **Summary**: `summary`.
- **Selectors**: Resilient fallbacks for author, time-attribute `<time datetime>`, category, sponsored/breaking markers, comment/share/related counts, summary.
- **Shared validator/normalizer** wired to `_canonical/`.

#### Changed

- Handler normalizes every record before pushing.
- `position` and `page` thread through to records.

### \[1.0.0] - 2025-05-20

#### Added

- Initial production release of the Brave News Scraper.
- `PlaywrightCrawler`-based architecture using Firefox (`apify/actor-node-playwright-firefox`).
- Modular project structure with `main.js`, `routes/`, `extractors/`, `filters/`, `schema/`, `utils/`, `errors/`.
- `.actor/actor.json`, `input_schema.json`, `dataset_schema.json`.
- `validate-datasets.js` for local dataset output validation.
- `Dockerfile` using `apify/actor-node-playwright-firefox` image.
- `README.md` with full Apify usage guide.