Academic Papers Scraper: OpenAlex, Crossref, arXiv & PubMed

Search scientific papers in four open databases at once and get one clean, de-duplicated list: title, DOI, authors with ORCID and institutions, venue, publisher, publication date, citation count, abstract, open-access PDF link, topics, MeSH terms, arXiv and PubMed IDs.

The same paper found in OpenAlex, Crossref, arXiv and PubMed appears once, with the best data of each source merged (sources tells you where it was found). You can also paste a list of DOIs (or point to a dataset / CSV) to enrich them in bulk, and schedule alerts for new papers on your topics.

Only official, open APIs. No Google Scholar scraping, no login, no proxies.

What does Academic Papers Scraper do?

Academic Papers Scraper searches scientific papers across OpenAlex, Crossref, arXiv and PubMed in one run and returns research papers de-duplicated by DOI, with authors, institutions, citation counts, abstracts and open-access PDF links. It also does bulk DOI lookup and new-paper alerts for a topic, useful for literature reviews and research monitoring.

How to use Academic Papers Scraper

1. Click Try for free on this page (a free Apify account is enough to start).
2. Fill in the input (see the examples below) or keep the prefilled example.
3. Click Start and wait for the run to finish.
4. Download the results from the Output tab as JSON, CSV, Excel or HTML, or read them with the Apify API.
5. Optional: create a Schedule to run it daily or weekly, and connect a webhook or an integration (Make, Zapier, n8n, Slack).

Who uses it

• Researchers and R&D teams: literature reviews, state of the art, follow a topic every week.
• Pharma, biotech and medtech: monitor PubMed and preprints for a molecule, disease or method.
• AI agents and RAG pipelines: fetch abstracts and open PDFs for a question (works as a tool through the Apify MCP server).
• Libraries, publishers and analysts: enrich DOI lists with citations, open-access status and affiliations.

Sources

Source	What it adds	Limit respected
OpenAlex	250M+ works, citation counts, institutions, topics, open-access status	Small shared budget without a key; add your free OpenAlex API key for your own daily budget
Crossref	The DOI registry of almost every publisher: venue, publisher, references count, licenses	Public pool 1 request/s; with your contact email, the faster "polite pool"
arXiv (via OpenAlex)	Preprints in physics, CS, math, quantitative biology, finance... with PDF. Searched through OpenAlex's arXiv source, so the arXiv servers are never hit	Same OpenAlex budget as above
PubMed (via Europe PMC)	Biomedical literature (PubMed/MEDLINE records), abstracts, MeSH terms, PMC full text. Searched through the Europe PMC REST API, which mirrors PubMed	1 request every 10 s (up to 1,000 records each), as Europe PMC's robots.txt asks

The arXiv and NCBI APIs are not called directly: their robots.txt disallows automated access, and this Actor respects robots.txt. Results are the same papers with the same arXiv IDs and PubMed IDs; small differences in ranking and freshness versus the arXiv / PubMed websites are possible.

Features

• Several queries per run; each one searched in every selected source.
• Dedupe and merge by DOI, arXiv ID, PubMed ID and title (a preprint and its published version are merged when they share a DOI or the exact title; two different DOIs are never merged).
• Filters: publication date range, open access only. Sort by relevance, newest or most cited.
• Bulk DOI lookup from a list, an Apify dataset or a CSV / Google Sheet.
• Only new mode with memory per alert name: schedule it and get only papers you have not seen.
• Crossref results are checked so every query term really appears in the title, abstract or venue (its search engine is very fuzzy).

Input example

{
  "queries": ["CRISPR base editing", "large language models protein design"],
  "sources": ["openalex", "crossref", "arxiv", "pubmed"],
  "fromDate": "2025-01-01",
  "sort": "newest",
  "maxResultsPerSource": 50,
  "onlyNew": true,
  "alertName": "weekly-crispr",
  "openAlexApiKey": "optional-free-key",
  "contactEmail": "you@example.com"
}

DOI enrichment only:

{ "dois": ["10.1038/nature14539", "https://doi.org/10.1126/science.1225829"] }

Output

{
  "title": "A Programmable Dual-RNA–Guided DNA Endonuclease in Adaptive Bacterial Immunity",
  "doi": "10.1126/science.1225829",
  "doiUrl": "https://doi.org/10.1126/science.1225829",
  "publicationDate": "2012-06-29",
  "year": 2012,
  "authorNames": "Martin Jínek; Krzysztof Chylinski; Ines Fonfara; M. Hauer; Jennifer A. Doudna; Emmanuelle Charpentier",
  "firstAuthor": "Martin Jínek",
  "authorCount": 6,
  "institutions": ["Howard Hughes Medical Institute", "University of California, Berkeley", "University of Vienna", "Umeå University"],
  "venue": "Science",
  "publisher": "American Association for the Advancement of Science",
  "type": "article",
  "citationCount": 17104,
  "isOpenAccess": true,
  "openAccessStatus": "green",
  "pdfUrl": null,
  "landingUrl": "https://doi.org/10.1126/science.1225829",
  "abstract": "Clustered regularly interspaced short palindromic repeats (CRISPR)/CRISPR-associated (Cas)...",
  "topics": ["CRISPR and Genetic Engineering", "RNA regulation and disease", "Cas9"],
  "keywords": ["CRISPR/Cas9", "sequence-specific DNA cleavage", "DNA double-strand breaks"],
  "meshTerms": [],
  "arxivId": null,
  "pmid": "22745249",
  "pmcid": null,
  "openAlexId": "W2045435533",
  "issn": "0036-8075",
  "language": "en",
  "authors": [{ "name": "Martin Jínek", "orcid": "0000-0002-7601-210X", "affiliations": ["Howard Hughes Medical Institute", "University of California, Berkeley"] }],
  "matchedQueries": [],
  "sources": ["openalex"],
  "fetchedAt": "2026-10-27T10:00:00.000Z"
}

Fields that a source does not have are null (for example citation counts only come from OpenAlex and Crossref). DOIs that no source knows are reported in a free row with an error field.

A run summary (requests, results per source, merged duplicates, notes) is saved as RUN_SUMMARY in the key-value store.

Pricing

Pay per event:

• USD 1.50 per 1,000 papers returned (USD 0.0015 each). Duplicates merged across sources count once.
• USD 0.01 per run (start).
• Free: DOIs not found, papers skipped in "only new" mode, and anything beyond your "max charge per run".

Tips and limits

• Without an OpenAlex key, OpenAlex shares one small daily budget per IP address; on busy cloud IPs it can be used up, and then the run continues with the other sources (see notes in the run summary). A free key fixes it.
• arXiv results depend on OpenAlex: if the OpenAlex budget runs out, arXiv is skipped too (a free OpenAlex key fixes it). Very recent arXiv preprints can take a few days to appear in OpenAlex.
• Europe PMC is queried once every 10 s, so many queries with PubMed selected take ~10 s each.
• Crossref and PubMed sometimes carry future "issue" dates (a 2027 print issue already online).
• Abstracts come from the sources; publishers may keep copyright on some of them. Use them according to your purpose (research, indexing) or turn them off.
• Citation counts are a snapshot from OpenAlex/Crossref and differ from Google Scholar.

Data and credits

Metadata from OpenAlex (CC0), Crossref (metadata openly available), arXiv records through OpenAlex ("Thank you to arXiv for use of its open access interoperability") and PubMed records through Europe PMC. This Actor is not affiliated with or endorsed by any of them.

Cost examples

Prices at the current pay-per-event rate: USD 0.01 per run and USD 1.50 per 1,000 papers (papers merged across sources count once; DOIs not found and papers skipped in only-new mode are free). You can set a maximum charge per run in the run options.

Job	Approximate cost
Test run: 1 query, 4 sources, 5 results each (measured: 19 papers after merging, 5 s)	about $0.04
Literature review: 3 queries, 200 papers	about $0.31
Enrich 1,000 DOIs from a CSV	about $1.51
Weekly alert in only-new mode (~30 new papers)	about $0.055 per run

Use it from your code, an integration or an AI agent

Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("latamdatatools/academic-papers-search").call(run_input={
  "queries": [
    "large language models protein design"
  ],
  "sources": [
    "openalex",
    "crossref",
    "arxiv",
    "pubmed"
  ],
  "maxResultsPerSource": 25,
  "sort": "newest"
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["year"], item["title"], item["doi"], item["citationCount"])

JavaScript / Node.js (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('latamdatatools/academic-papers-search').call({
  queries: [
    'large language models protein design'
  ],
  sources: [
    'openalex',
    'crossref',
    'arxiv',
    'pubmed'
  ],
  maxResultsPerSource: 25,
  sort: 'newest'
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map((i) => [i.year, i.title, i.doi]));

No code:

• Schedules: save your input as a Task and run it weekly from Apify Console (Schedules).
• Integrations: send the results to Google Sheets, Slack, email, Make, Zapier, n8n or Airbyte from the Integrations tab, or call your own webhook when a run finishes.
• AI agents: add it as a tool in Claude, Cursor, VS Code and other MCP clients through the Apify MCP server (https://mcp.apify.com), so the agent can search the scientific literature and cite real papers with DOIs.

FAQ

Can I call it from my own code or from an AI agent? Yes. Start runs and fetch results with the Apify API or the JavaScript / Python clients, or use it as a tool from Claude, Cursor and other MCP clients through the Apify MCP server (mcp.apify.com).

Can I run it automatically? Yes. Create a Schedule in Apify Console (for example every morning) and add a webhook or an integration to get the results where you need them.

Which export formats are available? JSON, JSONL, CSV, Excel (XLSX), XML, RSS and HTML table, from the Output tab or the API.

What does it cost? See the Pricing section above. You only pay for the events listed there.