# Academic Paper Search — OpenAlex, Crossref, arXiv & PubMed (`saliga/academic-paper-search`) Actor Search scholarly papers in OpenAlex, Crossref, arXiv and PubMed at once, or look up DOIs. One de-duplicated record per paper: title, DOI, year, venue, authors, abstract, citation counts, open access status and PDF link. Public open APIs only. $3 per 1,000 papers. - **URL**: https://apify.com/saliga/academic-paper-search.md - **Developed by:** [Salih Küçükkayış](https://apify.com/saliga) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 papers This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README Search scholarly papers in **OpenAlex, Crossref, arXiv and PubMed at once**, or look up a list of **DOIs**, and get **one de-duplicated record per paper in one schema**: title, DOI, year, venue, publisher, authors, abstract, **citation and reference counts**, **open access status** and a **PDF link** when a free copy exists. The same paper found in several sources is merged into one row, with each field taken from the source that is best at it. **$3 per 1,000 papers.** Searches that find nothing are free, DOIs that are not found are free, and a paper found again by a later search is not charged twice. No API key needed (an optional free OpenAlex key raises its daily limit), no scraping: only the public APIs of the four sources. ### Quick start Press **Start** with the example input (finishes in well under a minute): ```json { "queries": ["retrieval augmented generation"], "fromYear": 2024, "maxResultsPerQuery": 20, "sources": ["openalex", "arxiv"] } ``` Look up papers by DOI or arXiv ID: ```json { "dois": ["10.1038/s41586-021-03819-2", "https://doi.org/10.1145/3442188.3445922", "arXiv:2312.10997"] } ``` Most cited open access papers on a topic, with their abstracts, from all four sources: ```json { "queries": ["CRISPR base editing", "\"prime editing\" off-target"], "sources": ["openalex", "crossref", "arxiv", "pubmed"], "sort": "cited_by", "openAccessOnly": true, "fromYear": 2020, "maxResultsPerQuery": 100 } ``` One row per paper (example of the shape; the values are illustrative, not from a measured run): ```json { "title": "Retrieval-Augmented Generation for Large Language Models: A Survey", "doi": "10.48550/arxiv.2312.10997", "year": 2023, "publicationDate": "2023-12-18", "type": "preprint", "venue": "arXiv (Cornell University)", "publisher": "Cornell University", "authors": ["Yunfan Gao", "Yun Xiong", "Xinyu Gao", "..."], "authorCount": 10, "abstract": "Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination ...", "citedByCount": 1450, "referencesCount": 170, "isOpenAccess": true, "oaStatus": "green", "pdfUrl": "https://arxiv.org/pdf/2312.10997v5", "landingPageUrl": "https://arxiv.org/abs/2312.10997", "openalexId": "W4390XXXXXX", "arxivId": "2312.10997", "pmid": null, "pmcid": null, "sources": ["openalex", "arxiv"], "topics": ["Natural Language Processing Techniques", "Topic Modeling", "Advanced Graph Neural Networks"], "query": "retrieval augmented generation" } ``` ### What you get - **Four sources, one schema.** OpenAlex (250M+ works across all fields, citation counts, open access links), Crossref (publisher DOI metadata), arXiv (preprints with PDFs) and PubMed (biomedical literature). Every row has the same fields, whichever source found it; a field a source does not have is `null`. - **De-duplicated.** Results are merged when they share a DOI, then an arXiv ID (or a PubMed / OpenAlex ID), then the same normalised title and year (only when their DOIs do not disagree). An arXiv preprint whose metadata names the journal DOI is merged with the journal article. `sources` lists every API that returned the paper. - **Best field from the best source.** Citation and reference counts and open access status from OpenAlex; venue and publisher from Crossref; PDF link and abstract from arXiv; then the other sources as fallbacks. - **PDF links ready for text extraction.** `pdfUrl` is the arXiv PDF or the best open access PDF OpenAlex knows. It links to the original host; the Actor does not download or re-host PDFs. Feed it to [Document to Markdown](https://apify.com/saliga/document-to-markdown) to get Markdown and RAG chunks. - **Filters**: publication years, open access only, sort by relevance, most cited or newest. - **Citation graph (optional)**: the OpenAlex IDs of the works a paper cites (`referencedWorks`) and of up to 200 works that cite it (`citedByIds`). - **Bibliographic data only**: author display names for credit. No e-mail addresses, no affiliations, no ORCID iDs. - **Summary** in the key-value store record `SUMMARY`: results per search and source, papers merged or already written, DOIs not found or not written (spending limit or timeout), source errors, notes (such as the OpenAlex daily limit) and requests sent to each API. ### Input | Field | Default | What it does | |---|---|---| | `queries` | | Keyword searches. "Quoted phrases" stay phrases. On arXiv, queries written in arXiv syntax (`ti:`, `au:`, `cat:cs.CL`, `AND`) are passed as they are | | `dois` | | DOIs, doi.org links or arXiv IDs to look up | | `sources` | openalex, arxiv, crossref | Any of `openalex`, `crossref`, `arxiv`, `pubmed` | | `maxResultsPerQuery` | 25 | Results taken from **each source** for each search (max 1,000). After merging, a search gives between this number and this number × sources | | `fromYear`, `toYear` | | Publication years (searches only, not DOI lookups) | | `openAccessOnly` | false | Keep only papers that are free to read | | `sort` | relevance | `relevance`, `cited_by` or `newest` | | `includeAbstract` | true | Abstracts from OpenAlex and arXiv | | `includePublisherAbstracts` | false | Also abstracts from Crossref and PubMed (see *Good to know*) | | `includeReferences` | false | `referencedWorks`: OpenAlex IDs of the cited works | | `includeCitedBy` | false | `citedByIds`: OpenAlex IDs of up to 200 citing works, one extra request per paper | | `mailto` | | Your e-mail, sent to the APIs as the contact for the requests (optional). Add it if you use PubMed a lot | | `openalexApiKey` | | Your free OpenAlex API key (optional, stored as a secret). Raises the OpenAlex daily limit, see below | ### Output Every paper row has: `title`, `doi`, `year`, `publicationDate`, `type` (article, preprint, review, book-chapter, ...), `venue`, `publisher`, `authors` (display names, at most 100), `authorCount`, `abstract`, `citedByCount`, `referencesCount`, `isOpenAccess` (`null` when no source knows), `oaStatus` (gold, green, hybrid, bronze, diamond, closed), `pdfUrl`, `landingPageUrl`, `openalexId`, `arxivId`, `pmid`, `pmcid`, `sources`, `topics` (top 3 OpenAlex topics, or arXiv categories), `query`. DOI lookups add `inputDoi` (and `query` is `null`); the optional fields are `referencedWorks` and `citedByIds`. DOIs that are not valid or not found give a row with `inputDoi` and `error`; they are free. Export as JSON, CSV or Excel, or read the dataset through the API. ### What it costs | Run | Price | |---|---| | The example: 1 search, 20 results from OpenAlex and 20 from arXiv | at most $0.12 (fewer after merging) | | 10 searches, 25 results from each of 3 sources | at most $2.25 | | 1,000 DOIs looked up | $3.00 | | 1 search, 1,000 results from OpenAlex | $3.00 | You pay once per unique paper written to the dataset. Searches with no results, DOIs not found and papers already written for an earlier search in the same run are free. The run stops cleanly at your spending limit or shortly before its timeout, and keeps everything written so far. Many "Google Scholar" tools scrape Google's result pages. This Actor uses only the public APIs of the sources, returns one de-duplicated schema across four sources, charges per paper found (searches that find nothing are free), and gives you the open access PDF link ready for Markdown conversion. ### Good to know - **Speed.** Each API gets its own polite pace, and the sources are queried in parallel. arXiv asks for at least 3 seconds between requests; this Actor asks for up to 200 results per request, so 1,000 arXiv results take about 15 seconds of waiting. OpenAlex and PubMed also return up to 200 results per request. `includeCitedBy` adds one OpenAlex request per paper. - **Sorting** is done by each source. arXiv and PubMed have no citation counts, so for *Most cited* they return the most relevant results instead; the merged list of each search is then sorted by `citedByCount` (or by date for *Newest*). - **Open access only**: OpenAlex filters by its own open access data and PubMed by "free full text". Crossref has no open access flag, so Crossref results stay only when OpenAlex or arXiv also returned them as open, or their licence is Creative Commons. - **Publisher abstracts** (`includePublisherAbstracts`) come from Crossref and PubMed. Publishers supply them and some are copyrighted by the publisher, so they are off by default; check the publisher's terms before republishing them. - **Years** are publication years as each source records them; a preprint can be a year older than the journal version it was merged with. - **Very large author lists** (physics collaborations) are cut to the first 100 names; `authorCount` keeps the full number. - **Contact address.** Crossref and NCBI ask API users to identify themselves. Every request carries a descriptive User-Agent with a contact address: yours if you fill in `mailto`, otherwise the Actor's project address. Add your e-mail in `mailto` if you use PubMed a lot, so NCBI can reach you rather than the project if your searches cause a problem. - One source failing (for example a short outage) does not stop the run: the other sources still return results, and the error is in the log and in `SUMMARY`. ### OpenAlex daily limit OpenAlex gives every API key, and anonymous callers, a daily usage budget that resets at midnight UTC. Searches use the most of it, list and filter calls less, and fetching a single paper by its DOI is free (that is why a list of up to 50 DOIs is looked up one DOI at a time). Without your own key, your runs share the Actor's budget with everyone else who runs this Actor without a key, so on a busy day it can run out. When the budget is used up, OpenAlex answers "429 Too Many Requests". The Actor then retries once after a few seconds, and if the answer is still 429 it stops asking OpenAlex for the rest of the run (no long waits), carries on with the other sources, logs a warning and notes it in `SUMMARY` (`openalexBudgetReached`, `notes`). Papers found only in OpenAlex are then missing from that run. To get full OpenAlex results, create a free account at [openalex.org](https://openalex.org), copy your API key and paste it in **Your OpenAlex API key** (`openalexApiKey`). It is a secret input and is only sent to OpenAlex. The budget of a free key is about ten times the anonymous one (October 2026; OpenAlex sets and may change the numbers). ### Data sources - **OpenAlex** ([openalex.org](https://openalex.org)): data released under CC0. OpenAlex: Priem, J., Piwowar, H., & Orr, R. (2022). *OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts.* arXiv:2205.01833. - **Crossref** ([crossref.org](https://www.crossref.org)): metadata from the Crossref REST API, deposited by Crossref members. Abstracts in Crossref belong to their publishers. - **arXiv** ([arxiv.org](https://arxiv.org)): metadata from the arXiv API (released under CC0). Thank you to arXiv for use of its open access interoperability. Each paper links to arXiv; the PDFs stay on arXiv, under the licence the authors chose, and are not re-hosted. - **PubMed / NCBI** ([pubmed.ncbi.nlm.nih.gov](https://pubmed.ncbi.nlm.nih.gov)): data from the NCBI E-utilities, requested at no more than 3 requests a second. Some PubMed abstracts are copyrighted by their publishers. This Actor is independent. It is not affiliated with or endorsed by OpenAlex (OurResearch), Crossref, arXiv / Cornell University, NLM or NCBI. ### FAQ **Can I turn the papers into text for an LLM or RAG?** Yes. Take the `pdfUrl` values of the dataset and give them as `urls` to [Document to Markdown](https://apify.com/saliga/document-to-markdown): you get Markdown per paper and, optionally, RAG chunks with page numbers. Only open access PDFs have a `pdfUrl`. **Can an AI agent use it?** Yes, through the Apify MCP server. The input is a list of searches or DOIs with sensible defaults, for example "find 10 recent papers about retrieval augmented generation with PDFs". **Why do I get more rows than *Results per search*?** The number is per source. With three sources and 25 each you get up to 75 papers per search, fewer when the sources found the same papers. **Can I get Google Scholar results?** No. This Actor uses only public scholarly APIs. OpenAlex covers most of what Google Scholar indexes, with citation counts. **Can I get author e-mails or affiliations?** No. Author names are kept for bibliographic credit only. ### More from this developer - [Document to Markdown](https://apify.com/saliga/document-to-markdown): PDF, Word, PowerPoint, Excel and scans to clean Markdown for LLMs and RAG. - [Audio to Text](https://apify.com/saliga/audio-to-text): audio, video and podcast feeds to timestamped text and RAG chunks with Whisper. - [Bulk HTTP Requests](https://apify.com/saliga/bulk-http-requests): call any API once for every row of a CSV, JSON list or Apify dataset. - [UK Companies House](https://apify.com/saliga/uk-companies-house): UK company profiles and lists of new companies from the official API. - [Sitemap Extractor](https://apify.com/saliga/sitemap-extractor): every URL a website lists in its sitemaps. Code examples (curl, Python, Node.js, n8n) for this and the other Actors: [github.com/msalihk/apify-actor-examples](https://github.com/msalihk/apify-actor-examples). Questions or a missing feature: open an issue on the Actor page. # Actor input Schema ## `queries` (type: `array`): Keyword searches, one per line, e.g. retrieval augmented generation. Each search is sent to every selected source. "Quoted phrases" are kept as phrases. ## `dois` (type: `array`): DOIs (10.1038/s41586-021-03819-2), doi.org links or arXiv IDs (2312.10997). Each one found gives one paper; DOIs not found are listed with an error and are free. ## `sources` (type: `array`): Which open APIs to search. OpenAlex: 250M+ works with citation counts and open access links. Crossref: DOI metadata from publishers. arXiv: preprints with PDFs. PubMed: biomedical literature. ## `maxResultsPerQuery` (type: `integer`): How many results to take from each source for each search. Results from different sources are merged when they are the same paper, so a search gives between this number and this number × sources. ## `fromYear` (type: `integer`): Only papers published in or after this year (searches only, not DOI lookups). ## `toYear` (type: `integer`): Only papers published in or before this year (searches only). ## `openAccessOnly` (type: `boolean`): Keep only papers that are free to read. Crossref does not record open access, so its results are kept only when OpenAlex, arXiv or a Creative Commons licence shows the paper is open. ## `sort` (type: `string`): Order in which each source returns results (and of the merged list). arXiv and PubMed have no citation counts and use relevance for "Most cited". ## `includeAbstract` (type: `boolean`): Add the abstract when OpenAlex or arXiv has it. ## `includePublisherAbstracts` (type: `boolean`): Also use abstracts supplied by publishers through Crossref and PubMed (PubMed needs one more request per 200 papers). Some of these abstracts are copyrighted by the publisher: check before you republish them. ## `includeReferences` (type: `boolean`): Add referencedWorks: the OpenAlex IDs of the works the paper cites (needs OpenAlex as a source). The count, referencesCount, is always included. ## `includeCitedBy` (type: `boolean`): Add citedByIds: the OpenAlex IDs of up to 200 works that cite the paper, most cited first. One extra OpenAlex request per paper. The count, citedByCount, is always included. ## `mailto` (type: `string`): Sent to OpenAlex, Crossref and NCBI as the contact address of the requests, as they ask. Leave empty to use the Actor's project address; add your e-mail if you use PubMed a lot. ## `openalexApiKey` (type: `string`): OpenAlex gives every API key a free daily usage budget. Without your own key, runs share the Actor's budget, and when it is used up OpenAlex is skipped for the rest of the run (the other sources still answer). Add your free key from openalex.org for full OpenAlex results. ## Actor input object example ```json { "queries": [ "retrieval augmented generation" ], "sources": [ "openalex", "arxiv" ], "maxResultsPerQuery": 20, "fromYear": 2024, "openAccessOnly": false, "sort": "relevance", "includeAbstract": true, "includePublisherAbstracts": false, "includeReferences": false, "includeCitedBy": false } ``` # Actor output Schema ## `results` (type: `string`): One row per unique paper: title, year, venue, authors, citations, open access, PDF link, DOI. ## `fullResults` (type: `string`): Every field, including abstract, publisher, IDs (OpenAlex, arXiv, PubMed, PMC) and topics. ## `summary` (type: `string`): Results per search and source, duplicates merged, DOIs not found, source errors, requests per API. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "retrieval augmented generation" ], "sources": [ "openalex", "arxiv" ], "maxResultsPerQuery": 20, "fromYear": 2024 }; // Run the Actor and wait for it to finish const run = await client.actor("saliga/academic-paper-search").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["retrieval augmented generation"], "sources": [ "openalex", "arxiv", ], "maxResultsPerQuery": 20, "fromYear": 2024, } # Run the Actor and wait for it to finish run = client.actor("saliga/academic-paper-search").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "retrieval augmented generation" ], "sources": [ "openalex", "arxiv" ], "maxResultsPerQuery": 20, "fromYear": 2024 }' | apify call saliga/academic-paper-search --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,saliga/academic-paper-search" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/8XqudRafAwUxZp1WR/builds/c7Xe8ztOxohAbKAlV/openapi.json