Crossref Scraper · DOI Metadata, Authors, Journals & Citations

Crossref Scraper is a high-speed, light-footprint HTTP actor designed to extract scholarly DOI metadata, journal articles, books, preprints, citation counts, author affiliations, funding agency records, and open-access licensing information directly from the official Crossref REST API (api.crossref.org).

Maintained by reapX. Every row cites the Apify run that produced it — nothing is inferred, modelled or filled in, and a field absent from the source is absent from the row. The extracted archive for this source is browsable at reapx.dev/data/crossref-scraper/ and mirrored as an open dataset on Hugging Face and Kaggle. Questions: reapxdev@proton.me

Maintained by reapX. Public source data transformed into addressable entity records. Archive & API access guarantees. Contact reapxdev@proton.me.

---

Features & Capabilities

• Complete DOI Metadata: Extract digital object identifiers (doi), sanitized paper identifiers (paperId), article titles, publication dates, and publication types (journal-article, book-chapter, proceedings-article, posted-content, dataset).
• Authors & Affiliations: Collect full author lists, primary author affiliations, and author counts for every work.
• Journals, Publishers & ISSNs: Capture container titles, publisher names, volume, issue, page ranges, print/electronic ISSNs, and ISBNs.
• Citations & References: Retrieve citation counts (isReferencedByCount) and bibliography reference counts (referenceCount).
• Funding & Licensing: Track grant funding agency names (funderNames), funder registry DOIs (funderIds), open access status (isOpenAccess), and reuse license URLs (licenseUrl).
• Direct PDF & Document Links: Extract direct URLs to full text PDF documents and publisher landing pages whenever available.
• Flexible Filters & Deep Paging: Filter by keyword, author, title, journal, publisher member ID, date range, funder ID, license availability, or abstract presence.
• Pay-Per-Event Pricing: Charged strictly per returned record written to the default dataset. Empty searches, invalid queries, and rate-limited attempts cost zero.

---

⬇️ Input

The input configuration for Crossref Scraper allows you to filter and sort scholarly works across the entire Crossref catalog.

Input Parameters

Parameter	Type	Required	Default / Example	Description
query	string	No	10.1002	Free-text search terms for matching metadata across titles, abstracts, and full texts.
query_author	string	No	Doudna	Filter works by author given or family name.
query_title	string	No	deep learning	Restrict search terms specifically to work titles.
query_container_title	string	No	The CRISPR Journal	Filter works published in specific journals, proceedings, or books.
query_publisher	string	No	Wiley	Search works issued by specific publishers.
filter_type	string	No	journal-article	Restrict results by Crossref publication type.
filter_from_pub_date	string	No	2020-01-01	Filter works published on or after date (YYYY, YYYY-MM, or YYYY-MM-DD).
filter_until_pub_date	string	No	2026-12-31	Filter works published on or before date.
filter_has_license	boolean	No	true	Restrict output to works carrying explicit license metadata.
filter_has_abstract	boolean	No	true	Filter for works with indexed abstract text.
filter_has_funder	boolean	No	true	Filter for works with registered funding agency metadata.
filter_funder_id	string	No	10.13039/100000001	Filter by Crossref Funder Registry DOI (e.g. NSF or NIH).
filter_member	string	No	311	Filter by Crossref publisher member ID.
sort	string	No	relevance	Sort field (relevance, published, is-referenced-by-count, updated, created).
order	string	No	desc	Ordering direction (desc or asc).
mailto	string	No	reapxdev@proton.me	Email address sent in API headers for Crossref Polite Pool access.
max_results	integer	Yes	50	Maximum number of work records to return (1 to 10,000).

Worked Input Example

{
  "query": "crispr",
  "query_container_title": "The CRISPR Journal",
  "filter_type": "journal-article",
  "filter_has_license": true,
  "sort": "published",
  "order": "desc",
  "mailto": "reapxdev@proton.me",
  "max_results": 50
}

---

⬆️ Output

All extracted work metadata items are written directly to the run's default dataset.

Output Fields

Field	Type	Description	Example
paperId	string	Sanitized DOI with colons replaced by hyphens for entity page addressing.	10.1089-crispr.2018.29011.rba
doi	string	Raw canonical Digital Object Identifier.	10.1089/crispr.2018.29011.rba
title	string	Primary title of the paper, article, or chapter.	Cultivating CRISPR
publicationYear	integer	Four-digit publication year.	2018
publicationDate	string	Full ISO publication date or date string (YYYY-MM-DD or YYYY-MM).	2018-04-19
type	string	Crossref work type classification.	journal-article
publisher	string	Publishing house or issuing organization name.	SAGE Publications
containerTitle	string	Title of journal, book series, or conference volume.	The CRISPR Journal
volume	string	Journal volume designation.	1
issue	string	Journal issue designation.	2
page	string	Page range or article number.	99-100
issn	string	Comma-separated list of print and electronic ISSNs.	2573-1599, 2573-1602
isbn	string	Comma-separated list of ISBNs (for books/chapters).	978-0-12-345678-9
authors	string	Comma-separated author full names.	Rodolphe Barrangou
authorCount	integer	Total number of listed authors.	1
firstAuthorAffiliation	string	Institutional affiliation of the lead author.	Editor-in-Chief, The CRISPR Journal
isReferencedByCount	integer	Total citations reported in Crossref index.	15
referenceCount	integer	Total references in bibliography.	0
isOpenAccess	boolean	True if license or open access links are present.	true
licenseUrl	string	URL of reuse or text/data mining license.	http://www.liebertpub.com/nv/resources-tools/
funderNames	string	Comma-separated funding agency names.	National Institutes of Health
funderIds	string	Comma-separated funder registry DOIs.	10.13039/100000002
url	string	Canonical HTTP URL for resolving the DOI.	https://doi.org/10.1089/crispr.2018.29011.rba
pdfUrl	string	Direct URL to full text PDF document if available.	https://journals.sagepub.com/doi/pdf/10.1089/crispr.2018.29011.rba
abstract	string	Plain-text work abstract (HTML tags stripped).	This paper reviews recent developments in CRISPR gene editing...
subject	string	Subject categories or classifications.	Genetics, Biotechnology
language	string	Language code.	en

Worked Output Example

{
  "paperId": "10.1089-crispr.2018.29011.rba",
  "doi": "10.1089/crispr.2018.29011.rba",
  "title": "Cultivating CRISPR",
  "publicationYear": 2018,
  "publicationDate": "2018-04-19",
  "type": "journal-article",
  "publisher": "SAGE Publications",
  "containerTitle": "The CRISPR Journal",
  "volume": "1",
  "issue": "2",
  "page": "99-100",
  "issn": "2573-1599, 2573-1602",
  "isbn": null,
  "authors": "Rodolphe Barrangou",
  "authorCount": 1,
  "firstAuthorAffiliation": "Editor-in-Chief, The CRISPR Journal",
  "isReferencedByCount": 15,
  "referenceCount": 0,
  "isOpenAccess": true,
  "licenseUrl": "http://www.liebertpub.com/nv/resources-tools/text-and-data-mining-policy/121/",
  "funderNames": null,
  "funderIds": null,
  "url": "https://doi.org/10.1089/crispr.2018.29011.rba",
  "pdfUrl": "http://www.liebertpub.com/doi/pdf/10.1089/crispr.2018.29011.rba",
  "abstract": null,
  "subject": null,
  "language": "en"
}

---

How it works

1. Polite API Requests: Uses plain HTTP requests sent to Crossref REST endpoints with user-agent identification for routing through Crossref's high-speed Polite Pool.
2. Deep Cursor Pagination: Automatically paginates through search results using Crossref's cursor mechanism to retrieve large datasets reliably without loss of state.
3. Dual Identifier Generation: Emits both doi (the raw DOI string) and paperId (with colons replaced by hyphens). This ensures 100% entity page addressability across all datasets regardless of colons in DOIs.
4. Progressive Pushing: Pushes items in small batches to the default dataset as soon as they are processed.
5. Tiered Billing Safety: Charges strictly per returned item written to dataset. If the run budget cap (ACTOR_MAX_TOTAL_CHARGE_USD) is reached, execution stops cleanly without dropping completed records.

---

❓ FAQ

How are DOIs formatted in the output?

Crossref DOIs are emitted in two fields:

• doi: The exact, raw DOI string as registered in Crossref (e.g. 10.1051/0004-6361:20010000).
• paperId: The sanitized DOI with colons replaced by hyphens (e.g. 10.1051-0004-6361-20010000). This ensures valid identifier matching for entity pages.

How much does a run cost?

This actor uses Pay-Per-Event pricing. You are charged $0.002 per record returned (with automatic plan discounts down to $0.0008 at higher subscription tiers). Unsuccessful queries or empty results cost $0.00.

Why set the mailto field?

Providing an email address in mailto (default: reapxdev@proton.me) identifies your requests to Crossref's API servers. Crossref uses this to route traffic to their dedicated Polite Pool, providing significantly faster response times and higher rate limits.

Can I extract direct PDF download links?

Yes. When publishers provide full text PDF links in Crossref metadata, they are extracted into the pdfUrl field.

---

💬 Your feedback

We continuously improve our scrapers based on user feedback. If you discover a bug, request a new metadata field, or need custom extraction rules:

• Contact maintainers directly: reapxdev@proton.me
• Visit our asset portal: https://reapx.dev

---

Disclaimer: Unofficial - not affiliated with Crossref. Collects public data only. reapx. Contact reapxdev@proton.me.

🧪 Example input

A real, runnable configuration — this is an actual input this Actor has run with.

{
  "query": "climate change",
  "sort": "relevance",
  "max_results": 50,
  "mailto": "reapxdev@proton.me"
}

📄 Sample output

One real row from a real run of this Actor, unedited.

{
  "paperId": "10.1055/s-0036-1589185",
  "doi": "10.1055/s-0036-1589185",
  "title": "Synthesis of Exochomine",
  "publicationYear": 2016,
  "publicationDate": "2016-09-19",
  "type": "journal-article",
  "publisher": "Georg Thieme Verlag KG",
  "containerTitle": "Synfacts",
  "volume": "12",
  "issue": "10",
  "page": "1002-1002",
  "issn": "1861-1958, 1861-194X",
  "isbn": null,
  "authors": null,
  "authorCount": 0,
  "firstAuthorAffiliation": null,
  "isReferencedByCount": 0,
  "referenceCount": 0,
  "isOpenAccess": true,
  "licenseUrl": null,
  "funderNames": null,
  "funderIds": null,
  "url": "https://doi.org/10.1055/s-0036-1589185",
  "pdfUrl": "http://www.thieme-connect.de/products/ejournals/pdf/10.1055/s-0036-1589185.pdf",
  "abstract": null,
  "subject": null,
  "language": "en"
}

⚠️ Run outcomes and error handling

This Actor reports what happened in the run's status message, and it always keeps whatever it collected. These are the outcomes you can get and what each one means.

Outcome	What it means
Success	Rows were returned and you were charged record-returned at $0.002 per row.
No matches	The source returned nothing for your filters. Nothing is charged. Widen the date window or drop a filter.
Partial - source refused	The source rate-limited or refused some requests. The affected items are skipped and named in the log, and everything already collected is still pushed. A block never discards a run's work.
Rejected filter	The source itself rejected the filter combination. The run fails fast with the source's own reason and nothing is charged.

What is guaranteed either way

• Every row is pushed as it is built, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not.
• A field absent from the source is absent from the row. Nothing is inferred, modelled or filled in to make a row look complete.