Nature Search Results Scraper

Search the live Nature website with the same query, journal, article-type, subject, date-range, and sort controls available in its public search form. The Actor extracts server-rendered scholarly article cards and can enrich their verified 10.1038 DOIs through Crossref's public REST API.

Input

• searchQuery: keywords entered into Nature search.
• journal: one supported Nature Portfolio journal or all.
• articleTypes and subjects: optional current Nature filter codes; empty means all.
• dateRange: Nature's native relative date filter.
• sortBy: relevance, newest first, or oldest first.
• maxItems, maxPages, concurrency, retry, timeout, and optional proxy controls.
• includeCrossrefMetadata: adds DOI, citation/reference counts, volume/issue/pages, publisher, license, language, ISSNs, and Crossref subjects when available.
• fromYear and toYear: retained as legacy local filters. Nature does not expose custom year bounds in its standard search form, so completeness is reported explicitly when a bounded scan cannot cover every matching Nature page.

maxResults remains a backward-compatible alias for maxItems. The former citations sort is intentionally removed because Nature's current search form does not offer it; use relevance or publication date.

Output

Every record has a stable Nature article URL ID, contiguous position, title, open-access flag, the exact applied filter values, source page/URL, and timestamp. Available descriptions, authors, article type, publication date, journal, volume/page text, and verified Crossref metadata are included without null or placeholder fields.

The Actor validates final host, status, content type, body size, filter-preserving URL, total counts, result-card structure, Crossref JSON shape, and DOI identity before writing anything. It deduplicates by canonical Nature URL and writes atomically. CAPTCHAs, malformed payloads, and selector drift fail closed; genuine no-results searches complete with an empty dataset.

No raw Crossref payloads, cookies, headers, proxy URLs, contact email, or diagnostic placeholders are stored.

Verification

npm install
npm test
npm audit --omit=dev
apify validate-schema
apify run --purge --input-file .actor/input.json