# Research Papers Search API: arXiv, PubMed, Crossref, DOI (`solidfetch/research-papers`) Actor Search research papers, academic papers and scholarly articles in arXiv, PubMed, Crossref and OpenAlex at once. PubMed articles with abstracts, DOI, citation counts, open access papers and BibTeX per row, deduplicated. DOI lookup of references by title. $0.60-1 per 1,000. - **URL**: https://apify.com/solidfetch/research-papers.md - **Developed by:** [Solid Fetch](https://apify.com/solidfetch) (community) - **Categories:** Agents, AI, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.60 / 1,000 papers This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Research Papers Search: arXiv, PubMed, Crossref & OpenAlex API Research papers search across arXiv, PubMed, Crossref and OpenAlex: give a topic such as "protein folding transformer" and get academic papers deduplicated by DOI, with authors, year, abstract, citation count and an open-access PDF link where one exists. PubMed articles and their abstracts come in the same list, and every row carries the DOI, the citation count and a ready BibTeX entry. Give it the titles of your references (one per entry in `queries`, `maxResults` 1) and it works as a DOI lookup that fills in the DOI and metadata of each one, so check the match before you rely on it. Sort by citations for the most cited papers, or switch on open access only. A paper row costs $1.00 per 1,000 on the free plan, and failed sources and repeats are free. ```json { "query": "protein folding transformer", "maxResults": 10 } ``` One output row per paper (shortened from the full example under Output, a real answer for another topic): ```json { "title": "Retrieval-Augmented Generation for Large Language Models: A Survey", "year": 2023, "venue": "arXiv (Cornell University)", "doi": "10.48550/arxiv.2312.10997", "pdfUrl": "https://arxiv.org/pdf/2312.10997", "abstract": "Large language models (LLMs) demonstrate powerful capabilities, ...", "citationCount": 875, "isOpenAccess": true, "sources": ["openalex", "arxiv"] } ``` ### Research papers search: arXiv, PubMed, Crossref and OpenAlex API Search research papers and academic papers in four official scholarly APIs at once and get one deduplicated list: **OpenAlex**, **Crossref**, **arXiv** and **PubMed**. Every paper comes with its title, authors, year, venue, DOI, link, abstract where the source allows it, citation count, an open-access PDF link and the list of sources it was found in. A one-line `summaryText` makes each row easy for an AI agent to quote. It costs **$1.00 per 1,000 papers** on the free plan and **$0.60 per 1,000 on Gold and above**. Failed sources and repeats are free. ### Scholarly papers with the DOI, the abstract and BibTeX in every row Tools that scrape the results page of a search engine return a snippet, a title and a citation count, and they work through proxies and CAPTCHAs. This actor reads four official scholarly APIs instead, so every row has what a reference manager and an AI agent need: the **DOI**, the **full abstract** (where the source allows it), the **citation count**, an **open-access PDF link**, the paper's **type** (article, preprint, review, conference paper...), a **review/survey flag** and a ready-to-paste **BibTeX entry** for an export to your reference manager. | | Scrapers of search-result pages | This actor | |---|---|---| | DOI | not returned | yes, the DOI of the published version when one is known | | Abstract | a snippet | the abstract (OpenAlex, arXiv, Crossref, and PubMed when the record allows it) | | Citation count | yes (the search engine's own) | yes (OpenAlex, or Crossref) | | BibTeX | an extra request per paper, some scrapers | built from the row's own fields, in every row | | Blocks and CAPTCHAs | common, so proxies | none: official APIs | | Author profiles, h-index | some | no | It reads the public APIs of the four databases. There is no scraping, no Google Scholar and no full text of papers: only metadata, abstracts and links to open-access copies. If you need Google's own citation count or a Scholar author profile, this is not the tool. If you need the DOI, the abstract and a citation for a topic, it is. **Looking up a list of references.** The actor searches by words, not by DOI. To fill in the DOIs for a list of references, put each reference title in `queries` (up to 50 per run) and set `maxResults` to 1. Each row returns the best match with its DOI, authors, year, venue and BibTeX. A match is the closest paper by title, not a proof, so compare the title and year with your reference. ### What people use it for - **Literature search for an AI agent or a research assistant.** Pass a topic, get papers with abstracts and citation counts as compact JSON. Each row has a `summaryText` and consistent fields. - **Finding the open-access PDF.** `pdfUrl` and `openAccessUrl` point to free copies (arXiv, repositories, open journals). `openAccessOnly` returns only papers that are free to read. - **Seeing who cites what.** Sort by `citations` to get the most cited papers first, with counts from OpenAlex or Crossref. - **Keeping up with a field.** Sort by `date` with `yearFrom` to read the newest papers on a topic. - **Building a reading list across fields.** A medicine paper from PubMed, the preprint from arXiv and the publisher record from Crossref come back as one row with all three in `sources`. ### How to use it One search: ```json { "query": "retrieval augmented generation", "maxResults": 10 } ``` Several searches, recent papers only, free to read, most cited first: ```json { "queries": ["retrieval augmented generation", "agent memory", "LLM evaluation"], "yearFrom": 2023, "openAccessOnly": true, "sort": "citations", "maxResults": 20 } ``` Only the preprint server and the biomedical database: ```json { "query": "CRISPR base editing", "sources": ["arxiv", "pubmed"] } ``` A field name the actor does not know stops the run with a hint such as `"maxResult" (did you mean "maxResults"?)`, before any source is asked and before anything is charged. A misspelt setting cannot give a run that ignored it. ### Input | Field | What it does | |---|---| | `query` | One search text: keywords, a topic, a title or an author's name. Matched in title and abstract. | | `queries` | A list of search texts, up to 50 per run. Used together with `query`. | | `sources` | Any of `openalex`, `crossref`, `arxiv`, `pubmed`. Default: all four. | | `maxResults` | Papers returned per search after the sources are merged, 1 to 200. Default 10. | | `sort` | `relevance` (default), `date` (newest first) or `citations` (most cited first). OpenAlex, arXiv and PubMed sort natively; Crossref always contributes its most relevant matches, because its newest or most cited matches are mostly off topic. Everything is sorted again after the merge. | | `yearFrom`, `yearTo` | Publication year range. Either can be left empty. | | `openAccessOnly` | Only papers that are free to read. Crossref is skipped in this mode, because it has no open-access filter. | | `contactEmail` | Optional. Sent to Crossref, OpenAlex and NCBI as the contact of their polite pools. | | `openAlexApiKey` | Optional. A free OpenAlex key with its own daily budget (see below). | | `pubmedApiKey` | Optional. An NCBI key: 10 requests a second instead of 3. | ### Output One row per paper, in the order of the search. Fields are the same on every row (null where a source has nothing), so the CSV export works as is. An example (real data, shortened): ```json { "type": "paper", "query": "retrieval augmented generation", "rank": 1, "title": "Retrieval-Augmented Generation for Large Language Models: A Survey", "authors": ["Yunfan Gao", "Yun Xiong", "..."], "authorCount": 10, "year": 2023, "publicationDate": "2023-12-18", "venue": "arXiv (Cornell University)", "doi": "10.48550/arxiv.2312.10997", "url": "https://doi.org/10.48550/arxiv.2312.10997", "pdfUrl": "https://arxiv.org/pdf/2312.10997", "openAccessUrl": "https://arxiv.org/pdf/2312.10997", "abstract": "Large language models (LLMs) demonstrate powerful capabilities, ...", "abstractSource": "arxiv", "citationCount": 875, "isOpenAccess": true, "sources": ["openalex", "arxiv"], "paperType": "preprint", "isSurvey": true, "openalexId": "W4389984066", "arxivId": "2312.10997", "pmid": null, "pmcid": null, "bibtex": "@misc{gao2023retrieval,\n author = {Gao, Yunfan and Xiong, Yun},\n title = {Retrieval-Augmented Generation for Large Language Models: A Survey},\n howpublished = {arXiv (Cornell University)},\n year = {2023},\n doi = {10.48550/arxiv.2312.10997},\n eprint = {2312.10997},\n archivePrefix = {arXiv},\n url = {https://doi.org/10.48550/arxiv.2312.10997}\n}", "summaryText": "Retrieval-Augmented Generation for Large Language Models: A Survey (Yunfan Gao et al., 2023, arXiv (Cornell University)). 875 citations. Open access, PDF available. doi:10.48550/arxiv.2312.10997.", "retrievedAt": "2026-10-28T12:00:00.000Z" } ``` - **Authors** are the names as the databases publish them. The row holds the first 25 names, `authorCount` the total. There are no e-mail addresses and no affiliation addresses. - **Deduplication.** Papers are matched by DOI first, then by normalised title and year. The same paper from OpenAlex, arXiv and Crossref is one row. Every arXiv paper has a DOI (`10.48550/arxiv.`), so a preprint meets its OpenAlex record. A preprint and its journal version have different DOIs and the same title: when the year agrees, they are one row, and the DOI of the journal version is shown. - **`paperType`** is `article`, `preprint`, `review`, `conference-paper`, `book`, `book-chapter`, `thesis`, `dataset` or `other`, taken from the types the sources give (OpenAlex `type`, Crossref `type`, PubMed publication types; every arXiv record is a `preprint`). A paper found both as a preprint and in a journal is an `article`. `null` when no source says. **`isSurvey`** is `true` for a review or survey: a source calls it a review, or the title says "survey", "systematic review", "review of", "meta-analysis", "state of the art" and the like (a questionnaire survey of people is not counted). - **`bibtex`** is a BibTeX entry built from the row's own fields: `@article` for articles and reviews, `@inproceedings` for conference papers, `@misc` with `eprint` and `archivePrefix` for preprints. Authors are written "Family, Given", at most 25 followed by "and others". The characters `\ & % $ # _ { } ~ ^` are escaped; accented letters stay as UTF-8 (use biber, XeLaTeX or LuaLaTeX, or `\usepackage[utf8]{inputenc}`). Nothing is invented: volume, pages and publisher are not in the sources' search answers, so they are not in the entry. Check the entry before you cite it. - **`type`** is `paper` for the rows you pay for. `duplicate` is a paper an earlier search of the same run already returned, and `error` is a source that could not answer a search. Both are free. The `problems` view of the dataset lists just those. - **`citationCount`** comes from OpenAlex, or Crossref when OpenAlex does not have the paper. For a paper that only arXiv or PubMed found, the actor asks OpenAlex for its count by DOI or PubMed id, one extra request per 50 papers, so most rows have a number. The two sources count differently, so treat the number as a signal, not as a truth. It is `null` when no source knows the paper (a brand-new preprint), and `null` is not 0. If you leave OpenAlex out of `sources`, this extra lookup is not made. - **`isOpenAccess`** is `true` when any source says the paper is free to read, `false` when the sources say it is not, and `null` when nobody knows (Crossref and PubMed say nothing about it unless a PubMed Central copy exists). ### Where each field comes from, and the abstract rule | Source | Used for | Abstract | |---|---|---| | OpenAlex | title, authors, venue, year, citation count, open-access status and PDF link | Rebuilt from OpenAlex's word index. | | Crossref | publisher metadata, DOIs, citation count | The abstract the publisher deposited, as plain text. | | arXiv | preprints with an arXiv id, PDF link | The arXiv abstract. | | PubMed | biomedical papers with PMID, journal, PubMed Central link | Only when the PubMed record carries no copyright statement. | PubMed abstracts can be copyrighted by the publisher. The actor keeps an abstract out of every PubMed record that carries a copyright statement, and the abstract of such a paper is returned only if another source (OpenAlex, arXiv or Crossref) supplies one. `abstractSource` says where the text came from. There is no full text of any paper in the output, only links to free copies. ### Limits of the sources, said plainly - **OpenAlex.** At the time of writing, OpenAlex gives callers without a key a small daily budget (about $0.10, roughly a hundred list requests) shared by everyone on the same IP address, and a free key (openalex.org/settings/api) gets its own $1 a day. Apify runs share addresses, so a run without `openAlexApiKey` can find the OpenAlex budget used up. The run does not fail for that: the OpenAlex part returns a free `error` row with the reason, and the other three sources still deliver. For regular use, add a key. - **arXiv.** The arXiv API allows one request every 3 seconds, and the actor keeps to it. A run with 50 searches spends a few minutes waiting for arXiv, while the other sources work at the same time. - **Crossref and PubMed.** Crossref is asked at no more than 3 requests a second, PubMed at 3 a second (10 with `pubmedApiKey`). If you give `contactEmail`, it goes to their polite pools. Without it, the actor sends none. - **A source that fails twice in a row** is not asked again for the rest of the run, and the later searches get a free `error` row saying so. If every source fails for every search, the run fails and nothing is charged. - **Relevance** is a merge of the rankings of the sources (a paper several sources rank high comes first), with OpenAlex weighted most. PubMed ranks by word matches in medicine, so for a topic outside biomedicine its hits are weak and rank low. Narrow `sources` to what fits your field. - **Not covered.** Google Scholar, Semantic Scholar and Scopus are not used. Papers that no one has deposited with a DOI or an id in these four databases are not found. ### Pricing Pay per event: one `paper` event for each paper returned, **$0.001** on the free plan, $0.0009 on Bronze, $0.0008 on Silver and **$0.0006 on Gold and above**. You pay for the papers in the output, not for what the sources found before merging and cutting to `maxResults`. Repeats of a paper across searches and source errors are free rows. A run stops at the cost limit you set, with the papers saved so far. ### Questions **Can it fetch the full text?** No. It returns metadata, abstracts and links. Open the `pdfUrl` of an open-access paper if you need the text. **Why is a paper missing its abstract?** The source has none, or it is a PubMed record with a publisher copyright statement and no other source supplied one. **Why does the same paper appear once with two DOIs in different runs?** The DOI shown is the journal version when one is known, otherwise the preprint's. A search that finds only the preprint shows the arXiv DOI. **Does it restart safely?** Yes. Progress is saved after each search, and a restarted run does not save or charge the same papers again. Did this help? A review on this page, or an issue saying what is missing, helps other people find it. # Changelog This Actor's version history is a separate document: https://apify.com/solidfetch/research-papers/changelog.md # Actor input Schema ## `query` (type: `string`): What to search for: keywords, a topic, a title or an author's name. Words are matched in title and abstract. For arXiv, a phrase in double quotes is matched exactly. Use this or the list below, or both. ## `queries` (type: `array`): One search text per entry, up to 50 per run. A paper that matches several searches is returned and charged once; the repeat gets a free row. Same rules as the single search box above. ## `sources` (type: `array`): Which databases to ask. All four by default. OpenAlex covers all fields and has citation counts, Crossref has publisher metadata, arXiv has preprints in physics, maths and computer science, PubMed has biomedicine. ## `maxResults` (type: `integer`): Whole number from 1 to 200. The most papers returned for each search after the sources are merged. You pay for the papers returned, not for everything the sources found. ## `sort` (type: `string`): Relevance merges the rankings of the sources. Citations uses the counts from OpenAlex and Crossref; papers no source has a count for come last. ## `yearFrom` (type: `integer`): First publication year to include, a four-digit year such as 2023. Papers published earlier are dropped. Empty means no lower limit. ## `yearTo` (type: `integer`): Last publication year to include, a four-digit year such as 2025. Papers published later are dropped. Empty means no upper limit. ## `openAccessOnly` (type: `boolean`): Return only papers that are free to read. Crossref is skipped in this mode, because it has no open-access filter. ## `contactEmail` (type: `string`): Sent to Crossref, OpenAlex and NCBI as the contact of their polite pools, so they can write to you instead of blocking you. Leave empty to send none. Use a real address that you read, as an e-mail address such as name@example.com. ## `openAlexApiKey` (type: `string`): Without a key, OpenAlex gives a small daily budget shared by everyone on the same IP address, and the OpenAlex part of a run returns an error row once it is used up. A free key from openalex.org/settings/api has its own budget. ## `pubmedApiKey` (type: `string`): Raises the PubMed limit from 3 to 10 requests a second, which makes big runs faster. Free in your NCBI account settings. ## Actor input object example ```json { "query": "retrieval augmented generation", "queries": [ "retrieval augmented generation", "transformer language models survey" ], "sources": [ "openalex", "crossref", "arxiv", "pubmed" ], "maxResults": 20, "sort": "relevance", "yearFrom": 2023, "yearTo": 2025, "openAccessOnly": false, "contactEmail": "name@example.com" } ``` # Actor output Schema ## `papers` (type: `string`): Dataset table of papers matching the searches: title, authors, year, venue, DOI, links, abstract, citation count, open access status and BibTeX. Rows of type paper are the charged ones. ## `problems` (type: `string`): Dataset table of free rows only: sources that failed for a search and papers already returned by an earlier search of the run. ## `summary` (type: `string`): SUMMARY record in the key-value store: papers per source, source failures and charges for the run. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "retrieval augmented generation", "maxResults": 5 }; // Run the Actor and wait for it to finish const run = await client.actor("solidfetch/research-papers").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "retrieval augmented generation", "maxResults": 5, } # Run the Actor and wait for it to finish run = client.actor("solidfetch/research-papers").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "retrieval augmented generation", "maxResults": 5 }' | apify call solidfetch/research-papers --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,solidfetch/research-papers" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/XA0tEBz2KJNNS1OQQ/builds/L2xxvf1FJKckXShAu/openapi.json