Academic Papers Search: OpenAlex, Crossref, arXiv and PubMed in one run

Search research papers across OpenAlex, Crossref, arXiv and PubMed with one query. Each paper comes back as one flat row with title, authors, year, venue, DOI, arXiv ID, PubMed ID, citation count, open-access PDF link and abstract. A paper found in several sources is merged into a single row, deduplicated by DOI, arXiv ID and PMID, and the row lists which APIs returned it.

It uses the official open APIs, not Google Scholar pages, so there is no CAPTCHA, no blocking and no breakage when Google changes its HTML. A default run (50 papers, all four sources) finishes in about 10 seconds.

What you get

Field	Description
title, authors (list of names), year, publishedDate	Core bibliographic data
venue	Journal, conference or arXiv
doi, arxivId, pmid	Identifiers, normalised (lowercase DOI, arXiv ID without version)
citationCount	Highest count among OpenAlex and Crossref
isOpenAccess, pdfUrl	Free full text flag and link when known (null = unknown)
abstract	Plain-text abstract when any source has one
sources	Which APIs returned the paper, for example ["openalex","arxiv"]
url, scrapedAt, type	Landing page, UTC timestamp, always paper

Sample output

{
  "type": "paper",
  "title": "RAGAs: Automated Evaluation of Retrieval Augmented Generation",
  "authors": ["Shahul Es", "Jithin James", "Luis Espinosa-Anke"],
  "year": 2024,
  "publishedDate": "2024-01-01",
  "venue": "Conference of the European Chapter of the Association for Computational Linguistics",
  "doi": "10.18653/v1/2024.eacl-demo.16",
  "arxivId": "2309.15217",
  "pmid": null,
  "citationCount": 682,
  "isOpenAccess": true,
  "pdfUrl": "https://aclanthology.org/2024.eacl-demo.16.pdf",
  "abstract": "We introduce Ragas (Retrieval Augmented Generation Assessment), a framework for reference-free evaluation of ...",
  "sources": ["openalex", "arxiv"],
  "url": "https://doi.org/10.18653/v1/2024.eacl-demo.16",
  "scrapedAt": "2026-11-03T20:12:03Z"
}

Input

Field	Default	Notes
query	CRISPR base editing	Keywords or a phrase. Sent to every selected source.
sources	all four	openalex, crossref, arxiv, pubmed
yearFrom, yearTo	none	Publication year range
openAccessOnly	false	Keep only papers with a known free full text
sort	relevance	relevance (blended, multi-source hits first), citations, date
maxItems	50	Max papers returned (1 to 2000). You pay per paper.
onlyNew	false	Skip papers returned by an earlier run with the same query, sources, years and open-access setting. Schedule the actor to get new-paper alerts.
openAlexApiKey	none	Optional free key from openalex.org/settings/api
proxyConfiguration	none	Optional. Not needed for these open APIs.

{"query": "retrieval augmented generation", "sources": ["openalex", "arxiv"], "yearFrom": 2022, "sort": "citations", "maxItems": 100}

Pricing

Pay per event: $0.0005 per paper (about $0.50 per 1,000 papers). Rows that are not returned (failed sources, duplicates, filtered out, already seen) are never charged. Apify platform usage for a run this size is a fraction of a cent.

Use cases

• Literature reviews and reading lists across fields, from one query
• New-paper alerts: schedule a run with onlyNew and a webhook or email integration
• Citation-ranked shortlists of a topic (sort: citations)
• Feeding abstracts and open-access PDF links into RAG and summarisation pipelines
• Building datasets of papers by year, venue or source

FAQ

Is this a Google Scholar scraper? No. It queries OpenAlex, Crossref, arXiv and PubMed directly. Coverage is broad but not identical to Google Scholar (for example, no patents, court opinions or Scholar's "cited by" crawl).

How are duplicates handled? Papers are matched by DOI, arXiv ID and PMID, with a long normalised-title fallback for preprint and journal versions without a shared ID. Fields are merged: longest abstract, highest citation count, any known PDF link.

Why is citationCount sometimes 0 or null? arXiv and PubMed do not provide citations. Counts come from OpenAlex and Crossref, and differ between them and from Google Scholar.

Why do some rows lack an abstract? Crossref often has none and some publishers do not release abstracts to OpenAlex. Rows found by PubMed or arXiv usually have one.

What does sort: citations rank? The merged results of the per-source top matches, sorted by citation count. It is the most cited among the best matches, not a global most-cited list.

Limits

• Each source is asked for maxItems best matches, then merged. Crossref relevance can be noisy for short queries.
• arXiv allows one request every 3 seconds; large maxItems on arXiv take proportionally longer. Each source is capped at 1,000 results per run.
• OpenAlex without an API key shares a small daily budget per IP address. If OpenAlex rows go missing, the run summary (key OUTPUT) shows the source as failed; add a free OpenAlex key.
• isOpenAccess: null means unknown, not closed.
• Only public bibliographic metadata is returned.