Crossref Scholarly Scraper — DOIs, Citations & Journal Works API

Extract scholarly publication metadata from Crossref at scale — DOIs, citation counts, authors, journals, funders, and licenses — without any API key.

What does Crossref Scholarly Scraper do?

Crossref Scholarly Scraper connects directly to the Crossref REST API, the world's largest DOI registry covering 150+ million scholarly works from 50,000+ publishers. The actor supports three modes: works search (full-text keyword + filter queries with cursor-based deep pagination beyond 10,000 results), journal works (all publications from a specific ISSN), and DOI batch lookup (resolve a list of DOIs to full metadata). Each request is routed through Crossref's polite pool via a mailto parameter for higher rate limits. The actor extracts 17 structured fields per work — including is-referenced-by-count for citation counts, funder names, license URLs, and subject classifications — then pushes clean rows to an Apify dataset for immediate export. One run can retrieve hundreds to thousands of records with no authentication required.

Who is it for?

• Academic researchers who need bibliometric datasets for meta-analyses or systematic reviews
• Research librarians building citation databases or literature inventories for their institution
• Data scientists training NLP/ML models on scholarly text and metadata
• Journalists and analysts investigating research funding, publisher dominance, or topic trends
• Tech teams integrating DOI resolution or citation lookup into their SaaS or knowledge management tools

Use cases

• Download 1,000+ recent publications on "CRISPR gene editing" for a systematic literature review
• Pull all articles from a specific journal (by ISSN) to track citation growth over time
• Resolve a spreadsheet of 500 DOIs to full author/funder metadata for grant reporting
• Identify top funders and publishers in a research area using Crossref's filter API
• Build a citation graph by extracting citationCount and referencesCount across a topic

Why use Crossref Scholarly Scraper?

• No API key required — Crossref is a public, open metadata infrastructure; no registration needed
• 17 structured fields — DOI, title, authors, date, journal, publisher, type, citation count, reference count, subjects, URL, ISSN, volume, issue, page, license, funders
• Deep pagination via cursor — cursor-based pagination lets you retrieve far beyond the typical 10,000-item offset limit
• Three flexible modes — search by keyword/filter, pull a full journal, or resolve a DOI list
• Polite pool support — supply your mailto for Crossref's polite tier (higher throughput, no throttling)
• Export to CSV/JSON/Excel — all results available in Apify's dataset with one-click export

What data can you extract?

Each scholarly work returns the following fields:

Field	Type	Description
doi	string	Digital Object Identifier (e.g. 10.1038/s41586-023-06547-x)
title	string	Full title of the work
authors	string	Semicolon-separated "Given Family" author names
publishedDate	string	Publication date parts joined (e.g. 2023-09-13)
containerTitle	string	Journal or conference name
publisher	string	Publisher name (e.g. Springer Nature)
type	string	Work type: journal-article, book-chapter, proceedings-article, etc.
citationCount	string	How many other works cite this one (is-referenced-by-count)
referencesCount	string	Number of references in the work's bibliography
subject	string	Comma-separated subject classifications
url	string	Canonical DOI URL (https://doi.org/{doi})
issn	string	Journal ISSN(s)
volume	string	Journal volume number
issue	string	Journal issue number
page	string	Page range (e.g. 1234-1241)
license	string	License URL (e.g. Creative Commons URL)
funders	string	Semicolon-separated funder names

Example output record:

{
  "doi": "10.1038/s41586-023-06547-x",
  "title": "Highly accurate protein structure prediction with AlphaFold",
  "authors": "John Jumper; Richard Evans; Alexander Pritzel",
  "publishedDate": "2021-7-15",
  "containerTitle": "Nature",
  "publisher": "Springer Nature",
  "type": "journal-article",
  "citationCount": "23847",
  "referencesCount": "58",
  "subject": "Multidisciplinary",
  "url": "https://doi.org/10.1038/s41586-021-03819-2",
  "issn": "0028-0836, 1476-4687",
  "volume": "596",
  "issue": "7873",
  "page": "583-589",
  "license": "https://creativecommons.org/licenses/by/4.0",
  "funders": "Wellcome Trust; Google DeepMind"
}

How to use

Option A — Search by keyword (works mode)

Search the entire Crossref index by keyword, with optional date/type filters.

Steps:

1. Set mode to works
2. Enter your query (e.g. "deep learning image classification")
3. Optionally add a filter (e.g. from-pub-date:2022-01-01,type:journal-article)
4. Set maxResults (e.g. 500)
5. Run the actor

Input JSON:

{
  "mode": "works",
  "query": "deep learning",
  "filter": "from-pub-date:2023-01-01,type:journal-article",
  "maxResults": 300,
  "mailto": "you@example.com"
}

Option B — Fetch by journal ISSN (journalWorks mode)

Pull all works from a specific journal using its ISSN.

Steps:

1. Set mode to journalWorks
2. Enter the journal issn (e.g. 1476-4687 for Nature)
3. Optionally add a filter for date range
4. Set maxResults

Input JSON:

{
  "mode": "journalWorks",
  "issn": "1476-4687",
  "filter": "from-pub-date:2023-01-01",
  "maxResults": 200,
  "mailto": "you@example.com"
}

Option C — Resolve a DOI list (workByDoi mode)

Resolve specific DOIs to full metadata. Useful when you have a list of DOIs from another source.

Input JSON:

{
  "mode": "workByDoi",
  "dois": [
    "10.1038/s41586-021-03819-2",
    "10.1126/science.abm9891",
    "10.1016/j.cell.2022.11.001"
  ],
  "maxResults": 100,
  "mailto": "you@example.com"
}

Input parameters

Parameter	Type	Default	Description
mode	string	works	Scraping mode: works, journalWorks, or workByDoi
query	string	—	Keyword search for works mode
filter	string	—	Crossref filter (e.g. from-pub-date:2023,type:journal-article)
issn	string	—	Journal ISSN for journalWorks mode
dois	array	—	List of DOIs for workByDoi mode
maxResults	integer	300	Maximum records to return
mailto	string	contact@logiover.com	Email for Crossref polite pool (higher rate limits)
proxy	object	—	Apify proxy config (datacenter sufficient)

Full input example:

{
  "mode": "works",
  "query": "climate change adaptation",
  "filter": "from-pub-date:2020-01-01,has-full-text:true",
  "maxResults": 1000,
  "mailto": "researcher@university.edu",
  "proxy": { "useApifyProxy": true }
}

Output example

{
  "doi": "10.1016/j.neuron.2023.01.018",
  "title": "Cortical representations of abstract task structure during reinforcement learning",
  "authors": "Angela C. Nikou; Michael E. Frank",
  "publishedDate": "2023-3",
  "containerTitle": "Neuron",
  "publisher": "Elsevier BV",
  "type": "journal-article",
  "citationCount": "47",
  "referencesCount": "72",
  "subject": "Neuroscience",
  "url": "https://doi.org/10.1016/j.neuron.2023.01.018",
  "issn": "0896-6273",
  "volume": "111",
  "issue": "5",
  "page": "718-731.e8",
  "license": "https://creativecommons.org/licenses/by/4.0",
  "funders": "National Institutes of Health; Simons Foundation"
}

Tips for best results

• Use the polite pool: always supply your mailto for Crossref's polite pool — it dramatically increases rate limits and reliability
• Apply date filters: from-pub-date:2020-01-01 narrows results to recent publications; combine with until-pub-date for time windows
• Filter by type: add type:journal-article or type:book-chapter to the filter to focus on a specific publication type
• Use specific queries: "CRISPR therapeutic delivery 2023" returns more targeted results than just "CRISPR"
• Combine filter flags: Crossref supports multiple filters comma-separated, e.g. from-pub-date:2022,has-orcid:true,has-full-text:true
• DOI mode for known records: if you have DOIs from another dataset, workByDoi mode gives the most complete metadata per record
• ISSN mode for journal audits: use journalWorks with a date filter to track a journal's publication history
• Check null fields: not all Crossref records have every field — license and funders are only present in about 30–60% of records
• Large runs: for 5,000+ results, ensure your Apify memory is set to at least 1 GB; cursor pagination handles the depth automatically

Integrations

• Google Sheets: use the Apify → Google Sheets integration to push results directly to a spreadsheet for collaboration
• Slack: configure a webhook to receive a notification when your scholarly scrape completes
• Zapier / Make: trigger downstream workflows — e.g. create Notion database entries for each paper, or email a summary CSV
• Webhooks: configure an Apify webhook to POST dataset results to your internal API or ETL pipeline
• Schedule: run on a weekly cron to monitor new publications on your research topic automatically

API usage

cURL:

curl -X POST "https://api.apify.com/v2/acts/logiover~crossref-scraper/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"works","query":"deep learning","maxResults":200}'

Node.js:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('logiover~crossref-scraper').call({
  mode: 'works',
  query: 'machine learning drug discovery',
  maxResults: 500,
  mailto: 'you@example.com',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python:

from apify_client import ApifyClient

client = ApifyClient('YOUR_TOKEN')
run = client.actor('logiover~crossref-scraper').call(run_input={
    'mode': 'works',
    'query': 'CRISPR gene editing',
    'filter': 'from-pub-date:2022-01-01',
    'maxResults': 300,
    'mailto': 'you@example.com',
})
items = client.dataset(run['defaultDatasetId']).list_items().items

Use with AI agents (MCP)

Crossref Scholarly Scraper is available as an MCP tool for AI coding agents. Connect it to Claude, GPT-4, or any MCP-compatible agent and prompt: "Fetch the 200 most-cited journal articles on 'transformer attention mechanism' published after 2021, then summarize the top funders and publishers." The actor will run, return structured JSON, and your agent can proceed with analysis — no API key management required.

FAQ

Does this require an API key or login?

No. Crossref's REST API is completely public and keyless. The mailto parameter is optional but recommended — it places you in Crossref's polite pool for higher rate limits.

How many results can I get in one run?

Virtually unlimited. Cursor-based pagination bypasses Crossref's 10,000-item offset limit. In practice, you can retrieve hundreds of thousands of records for broad queries.

What sources does Crossref cover?

Crossref covers 150+ million scholarly works from 50,000+ publishers, including journals, conference proceedings, books, preprints, datasets, and reports. Coverage is strongest for journal articles published after 2000.

Why do some fields return null?

Not every publisher submits complete metadata. Fields like license, funders, abstract, issn, volume, and page are only present when publishers deposit them. Citation counts are available for most journal articles.

What does the filter parameter support?

Crossref supports many filter flags: from-pub-date, until-pub-date, type (journal-article, book-chapter, etc.), has-full-text, has-orcid, has-references, has-funder, member (publisher member ID), and more. See Crossref filter docs.

Can I get zero results?

Yes, if your query or ISSN doesn't match any Crossref records. Try broadening the query, removing filters, or checking the ISSN format (e.g. 1476-4687 not 14764687).

How do I export results?

In the Apify dataset view, click Export and choose CSV, JSON, XLSX, XML, or RSS. You can also use the Apify API to fetch results programmatically.

How fast does it run?

Crossref's polite pool allows ~50 requests/second. Each page fetches 100 items, so 300 results takes roughly 3-6 seconds of API time. A run of 5,000 results typically completes in under 2 minutes.

Is there a cost?

You pay Apify platform compute costs (typically $0.005–$0.02 per 1,000 results). There is no per-use charge from Crossref.

How often is Crossref data updated?

Crossref updates in near real-time as publishers deposit metadata. New publications typically appear within hours to days of being registered.

Can I filter by open access only?

Yes — use filter: "has-license:true" to return only works with a license deposited (usually open access). Combine with from-pub-date for recent OA literature.

What's the difference between citationCount and referencesCount?

citationCount (Crossref field: is-referenced-by-count) = how many other Crossref works cite this paper. referencesCount = how many references are in this paper's bibliography.

Is it legal?

Crossref is an open scholarly infrastructure funded by member publishers to make DOI metadata freely available. The Crossref REST API is publicly documented and intended for programmatic access. This actor accesses only the public, unauthenticated endpoints, respects rate limits via the polite pool, and does not bypass any access controls. Academic metadata scraped here contains factual bibliographic information with no personal data concerns. Always review Crossref's terms of service and your institution's data use policies before using results in published research.

Related scrapers

• logiover/pubmed-scraper — PubMed/MEDLINE articles with abstracts and MeSH terms
• logiover/arxiv-scraper — arXiv preprints by category, keyword, or author
• logiover/github-activity-scraper — GitHub repository events, commits, and contributor activity
• logiover/cve-advisory-scraper — CVE advisories and security vulnerability data from NVD