Academic Papers Search (unofficial PubMed, arXiv, Crossref)

Unofficial Apify Actor that searches public scholarly APIs and returns one row per paper. It is not affiliated with NCBI, the National Library of Medicine, arXiv, Cornell University, Crossref, or OpenAlex.

The default input searches PubMed, arXiv, and Crossref for machine learning and asks each source for 3 papers. OpenAlex is off unless you paste your own API key.

What you get

Each dataset row is one work after DOI de-duplication:

Field	Meaning
source	pubmed, arxiv, crossref, or openalex
id	PMID, arXiv id, DOI, or OpenAlex id
title	Work title
authors	Up to 100 author names
abstract	Abstract when the source license allows it, otherwise null
abstractStatus	included, omitted-copyright, or not-available
journal	Journal or venue
publishedDate	YYYY-MM-DD, YYYY-MM, or YYYY
doi	Lowercase DOI, or null
url	Landing page
pdfUrl	Open PDF link when one is known
license	Crossref Creative Commons URL when the work lists one this Actor can rely on
alsoFoundIn	Other selected sources that had the same DOI

Rows that share a DOI are merged. The primary id prefers PubMed, then Crossref, then OpenAlex, then arXiv. The abstract prefers arXiv, then OpenAlex, then a Crossref record whose license allows it. The PDF link prefers arXiv, then OpenAlex, then Crossref.

Works with no DOI are kept as separate rows. The same paper can appear twice when one of the records has no DOI.

Input

Field	Default	Role
query	machine learning	Keyword text sent to every selected source
sources	PubMed, arXiv, Crossref	Sources to query
publishedAfter, publishedBefore	empty	Inclusive date window (YYYY-MM-DD)
maxResultsPerSource	3	1–50 papers from each source before de-duplication
onlyNewSinceLastRun	false	Skip papers this monitor already returned
monitorKey	default	Memory name for only-new mode
contactEmail	plusonecares@gmail.com	NCBI email and Crossref mailto
ncbiApiKey	empty	Optional. Your NCBI key. Secret.
openalexApiKey	empty	Required for OpenAlex. Your key. Secret.

The default is the maintainer's address (plusonecares@gmail.com). You can replace it with your own. NCBI and Crossref use it to contact the caller if a run misbehaves. It is not used for anything else.

PubMed, Crossref, and OpenAlex filter on publication date. arXiv filters on submission date, which is the date the API exposes for this purpose.

Only new since last run

Turn onlyNewSinceLastRun on for a saved search. The Actor stores DOI and source ids in the named key-value store academic-papers-search-seen, under a record derived from monitorKey. The next run with the same key skips those works and reads up to 3 result pages per source to fill maxResultsPerSource with papers it has not returned yet.

Leave only-new mode off for a one-off search. The default input leaves it off, so a repeated run of that input still returns papers. Use a different monitorKey for each saved search. The memory keeps the newest 10,000 ids per key.

Output example

{
  "source": "pubmed",
  "id": "111",
  "title": "Example title returned by the search",
  "authors": ["Ada Lovelace", "Alan Turing"],
  "abstract": "Short abstract from the arXiv record that shared this DOI.",
  "abstractStatus": "included",
  "journal": "Example Journal",
  "publishedDate": "2024-05-01",
  "doi": "10.1000/example",
  "url": "https://pubmed.ncbi.nlm.nih.gov/111/",
  "pdfUrl": "https://arxiv.org/pdf/2401.00001",
  "license": null,
  "alsoFoundIn": ["arxiv"]
}

A PubMed-only row has abstract: null, abstractStatus: "omitted-copyright", and pdfUrl: null. An arXiv row includes the abstract and a PDF link on arxiv.org.

Pricing

Pay per event:

Event	When it is charged	Suggested price
apify-actor-start	Once per run, by the platform (once per GB of memory). The Actor does not call Actor.charge for it.	$0.005
paper-result	One de-duplicated paper written to the dataset	$0.002

In Apify Console, set the pricing model to pay per event, add paper-result as the primary event at $0.002, and remove the synthetic dataset-item event if the Console adds one. This Actor calls Actor.charge for paper-result only. It stops pushing when a charge returns that the run's max charge has been reached. The row that filled the budget is kept. Later rows are not written and are not marked as already seen.

The default input requests at most 9 papers before de-duplication, so a full default run is at most $0.023 ($0.005 start + 9 × $0.002). Set the run's maximum charge to at least $0.025 if you want every default-input row.

There is a $0.005 start fee per run, charged by the platform. Platform usage is not billed to you.

Limits and upstream rules

• At most 50 papers per source per run, and at most 3 pages per source in only-new mode.
• NCBI E-utilities: at most 3 requests per second without an API key, and at most 10 with your key. This Actor waits 0.35 seconds between NCBI calls without a key and 0.11 seconds with one. Every call sends tool=academic-papers-search and your contactEmail.
• arXiv API: at most one request every 3 seconds, one connection at a time. The terms of use are at https://info.arxiv.org/help/api/tou.html.
• Crossref polite pool: every call sends mailto. On 23 October 2026 the polite pool reported a limit of 3 requests per second. This Actor waits 0.4 seconds between Crossref calls.
• OpenAlex is requested only with your API key, sent as a bearer token. Calls are spaced 0.2 seconds apart, under the free-key limit of 30 requests per second.
• Failed calls retry up to 3 times on HTTP 429 and 5xx, honoring Retry-After up to 30 seconds. A 4xx response other than 429 fails that source immediately.
• If one source fails, the run still returns papers from the sources that succeeded. If every selected source fails, the run fails.
• PDF files are not downloaded or stored. pdfUrl is a link.
• PubMed rows leave pdfUrl empty. The PMC OA Web Service was retired on 25 August 2026, so this Actor no longer asks NCBI which PubMed articles have an open PDF. A DOI match can still attach an arXiv, OpenAlex, or Crossref open PDF.
• Apify runs share outbound addresses. Keep result caps modest so NCBI's per-address limit is respected. A personal NCBI API key raises the allowance for your key.

Publish the Actor with limited permissions (the Apify default). Actors that require full permissions are omitted from the Apify MCP index.

Abstracts, PDFs, and attribution

• PubMed. Titles, authors, journal, date, PMID, and DOI come from NCBI E-utilities ESearch and ESummary. Abstracts are not copied. NLM says PubMed abstracts may be protected by copyright and that people who reproduce them must follow the copyright holder's terms. NCBI's disclaimer has to be visible to users of software that calls E-utilities: https://www.ncbi.nlm.nih.gov/home/about/policies/. Usage guidelines: https://www.ncbi.nlm.nih.gov/books/NBK25497/.
• arXiv. Title, authors, abstract, categories, and identifiers are descriptive metadata dedicated under CC0 by the arXiv API terms. The PDF link points at arxiv.org. This Actor does not store or serve e-print files. arXiv is not the copyright holder of the PDFs.
• Crossref. Bibliographic metadata is factual data Crossref distributes for reuse. Abstracts stay in the dataset only when the work record lists CC0, CC BY, CC BY-SA, or CC BY-ND. Non-commercial licenses (CC BY-NC and its variants) and unlicensed abstracts are dropped. A PDF link is included only for those same commercial Creative Commons licenses. Editors fill the author list when a work has no authors. A record with no journal title uses the Crossref work type, such as book or monograph, as the venue. Some book records are deposited with neither authors nor editors; those rows have an empty authors array. Attribute Crossref when you republish the metadata: https://www.crossref.org/documentation/retrieve-metadata/.
• OpenAlex. Included only with your key. OpenAlex describes its data as CC0 and provides abstracts as an inverted index. This Actor reconstructs that index. Attribute OpenAlex: https://openalex.org.

license on a row is the Crossref Creative Commons URL for that work when one was used. An empty license on an arXiv or OpenAlex abstract means the text is included under that source's CC0 metadata policy, which is not a license for the PDF.

Run locally

Python 3.12.

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Save this as INPUT.json in the working directory:

{
  "query": "machine learning",
  "sources": ["pubmed", "arxiv", "crossref"],
  "maxResultsPerSource": 3,
  "onlyNewSinceLastRun": false,
  "monitorKey": "default",
  "contactEmail": "plusonecares@gmail.com"
}

$python -m src

Rows are written to storage/datasets/default/. On Apify, apify push from this directory builds the Actor. Set the pay-per-event price in the Console after the push. This repository does not contain an Apify token.

pip install -r requirements-dev.txt
pytest

Unit tests mock HTTP. They do not call PubMed, arXiv, Crossref, or OpenAlex.

Known limits

Publisher dates in Crossref are sometimes wrong, including dates in the far future. The default sort on each source is that source's relevance ranking, so those records do not float to the top unless they are also relevant.

DOI de-duplication misses a preprint and a journal article when one of them has no DOI yet.

Only-new memory lives in your Apify account's key-value store. Deleting academic-papers-search-seen starts that memory over. A run that stops on the charge limit remembers only the papers it pushed.

OpenAlex usage draws on the budget of the key you provide. This Actor does not call OpenAlex without a key, because keyless traffic shares a small public budget.

NCBI can block an address that exceeds its rate or omits a registered tool and email. Passing tool and email on each call is what this Actor does. Registering those values with NCBI is a separate step the operator takes if NCBI asks for it.