Nature Journal Search Scraper

Search the live Nature website with the same query, journal, article-type, subject, date-range, and sort controls available in its public search form. The Actor extracts server-rendered scholarly article cards and can enrich their verified 10.1038 DOIs through Crossref's public REST API. Nature's observed search pagination uses 50 results per page; the Actor records the reported total result count, derived page count, and page capacity with each record.

Input

• searchQuery: keywords entered into Nature search.
• journal: one supported Nature Portfolio journal or all.
• articleTypes and subjects: optional current Nature filter codes; empty means all.
• dateRange: Nature's native relative date filter.
• fromYear and toYear: a custom publication-year range. When both are set they are sent as Nature's own date_range=YYYY-YYYY search parameter, which is exactly what Nature's advanced search form submits, so the filter is applied by Nature rather than after the fact; dateRange is then not used. A single bound has no native parameter and stays a local post-filter on the publication year.
• sortBy: relevance, newest first, or oldest first.
• maxItems, maxPages, concurrency, retry, timeout, and optional proxy controls. maxPages defaults to 10 and supports up to 384 pages, matching the observed query's pagination.
• includeCrossrefMetadata: adds DOI, citation/reference counts, volume/issue/pages, publisher, license, language, ISSNs, and Crossref subjects when available.
• includeArticleSubjects: reads each written record's own Nature article page and records its published subject terms (dc.subject metadata and related-subject links), unioned with Crossref subjects. Nature's search result cards carry no subject terms, so this is the only place the target publishes them. It is one bounded, fail-soft request per record; a record is still written without subjects if its article page cannot be read, and the counts are reported in the run summary.

maxResults remains a backward-compatible alias for maxItems. The former citations sort is intentionally removed because Nature's current search form does not offer it; use relevance or publication date.

The complete input contract, including supported filters, defaults, and limits, is in .actor/input_schema.json.

Output

Every record has a stable Nature article URL ID, contiguous position, title, query-level pagination metadata (searchTotalResults, searchTotalPages, and resultsPerPage), open-access flag, the exact applied filter values (including a custom fromYear/toYear range and the dateRange value actually sent), source page/URL, and timestamp. Available descriptions, displayed authors, article type, publication date, journal, volume/page text, Nature subject terms, and verified Crossref metadata are included without null or placeholder fields. The dataset schema declares 39 meaningful fields; source-specific and Crossref fields are omitted when they are unavailable.

The Actor validates final host, status, content type, body size, filter-preserving URL, total counts, result-card structure, Crossref JSON shape, DOI identity, and the article-page subject surface before writing anything. It deduplicates by canonical Nature URL and writes atomically. CAPTCHAs, malformed payloads, and selector drift fail closed; genuine no-results searches complete with an empty dataset. Subject enrichment is the only fail-soft step: it is supplementary data, so one unreadable article page drops only that record's subjects and is counted in the run summary instead of discarding already-verified search results.

No raw Crossref payloads, cookies, headers, proxy URLs, contact email, or diagnostic placeholders are stored.

The full dataset field schema, including source-specific and Crossref fields, is in .actor/dataset_schema.json.

Live page observation

The public page at https://www.nature.com/search?q=CRISPR%20gene%20editing was inspected in Chrome. It displayed 19,167 results, 50 results per page, and 384 pages, with Journal, Article type, Subject, and Date filters and relevance/date sorting. Visible cards exposed titles and links, descriptions, displayed authors, article types, Open Access labels, publication dates, journal names, and volume/page text. One observed card was “Boosting genome editing in perennial plants by CRISPR-Combo mediated morphogenic gene activation” (Research, Open Access, 05 Aug 2026, Nature Communications, P: 1–16); another Research card was dated 16 Sept 2026 in Nature Structural & Molecular Biology. These are browser observations used to align the extractor and schema; they are not Actor output or a sample dataset.

Two further live observations set the contract for the fields that are not on the result cards:

• Year range. The visible result cards and the sidebar date facet expose only relative ranges (today, last_7_days, last_30_days, last_year, last_2_years, last_5_years); start_year/end_year are ignored by /search. Nature's advanced form posts those two fields to https://www.nature.com/search/submit, which answered HTTP 200 and redirected to https://www.nature.com/search?q=CRISPR&journal=nature&article_type=research&order=relevance&date_range=2020-2022 with 438 results instead of the 1,708 unfiltered ones, so the Actor sends that same date_range value.
• Subjects. The result cards carry no subject terms: across 50 cards the only data-test hooks were article-description, article.type, author-list, journal-title-and-link, open-access, and volume-and-page-info, with no subject/keyword text and no article-level JSON-LD payload. Nature's article pages do publish them, e.g. https://www.nature.com/articles/s41586-026-11113-2 carried meta[name="dc.subject"] values “Genetics research”, “Genome-wide association studies”, “Anaemia”, and “Haematopoietic stem cells” plus li[data-test="related-subject-item"] links to /subjects/<slug>. Crossref returns an empty message.subject for these Nature DOIs, so the article page is the only source.

How to use

Enter a search query, journal, and optional filters in the Apify Input tab. Start the Actor and review normalized article records in the Output dataset.