Yandex News Scraper

Search the current public Yandex News experience, which now redirects to and is served by Dzen News. Version 3 navigates directly to the supported public search surface with domcontentloaded, waits for current semantic article cards, and extracts destination links, headlines, snippets, publishers, source publication labels, and normalized dates. It does not store cookies, SSO parameters, raw responses, challenge pages, or fabricated ad/breaking flags.

Modes and filters

• Single query with query.
• Fair batch mode with up to 20 queries.
• Single result with maxItems: 1.
• Global and per-query limits.
• Past day/week/month and custom inclusive publication dates.
• Include/exclude destination domains, publishers, and title/snippet keywords.
• Relevance, newest, oldest, publisher, or title sorting.
• Yandex regionCode, language/country provenance, direct access, Residential proxy, or custom browser proxies.

The current search page exposes a finite server-rendered result set and no verified pagination link or content API. The Actor therefore does not invent pagination or repeatedly scroll a static page. Use multiple queries to broaden coverage.

Output

Every record has a stable URL-derived ID, rank/query provenance, canonical destination URL with tracking parameters removed, domain, headline and length, optional snippet and length, publisher and logo, normalized/raw publication time, article age, source/migration labels, matched queries, and scrape timestamp. Values missing from the public card are omitted instead of stored as null, empty strings, or empty arrays.

Safety and reliability

The Actor uses coherent Chrome fingerprints, persistent cookies per bounded session, optional authorized proxies, temporary-failure retries, resource blocking verified against metadata completeness, atomic dataset writes, and fail-closed challenge detection. GOOGLE_SERP is rejected because it supports specified Google HTTP Search/Shopping traffic, not Dzen/Yandex News browser navigation. The Actor does not solve or bypass CAPTCHAs or authentication.

Local validation

npm ci
npm test
node validate-datasets.js
apify run --purge --input-file qa-inputs/local-single.json

Representative single, batch, filtered, empty-result, low-concurrency, invalid-input, proxy-rejection, and cloud inputs are in qa-inputs/.