# arXiv & PubMed Paper Search: Abstracts, DOI, MeSH, PDF Links (`manageable_gully/arxiv-pubmed-paper-search`) Actor

Search research papers on arXiv and PubMed through their official APIs: title, abstract, authors, dates, DOI, journal, arXiv categories, MeSH terms and PDF links. Keyword, category, date or ID/DOI lookup, duplicates merged, only-new monitoring. $1 per 1,000 papers.

- **URL**: https://apify.com/manageable_gully/arxiv-pubmed-paper-search.md
- **Developed by:** [WebIntel Tools](https://apify.com/manageable_gully) (community)
- **Categories:** Education, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 papers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## arXiv & PubMed Paper Search: Abstracts, Authors, DOI, MeSH & PDF Links

**Search research papers on arXiv and PubMed in one run** and get clean, structured metadata: **title, abstract, authors, publication dates, DOI, journal, volume/issue/pages, arXiv categories, MeSH terms, keywords, publication types** and links to the **abstract page, PDF, PubMed and PubMed Central**. Search by keyword, arXiv category and date range, or look up a list of **arXiv IDs, PMIDs, PMCIDs and DOIs**. Papers that appear in both databases are merged, and the **only-new mode** turns any search into a daily or weekly literature alert. Data comes from the **official arXiv API and NCBI E-utilities** (plus Crossref for other DOIs), so there is no fragile HTML scraping. **$1 per 1,000 papers.**

### What you can use it for

- **Literature reviews and systematic searches:** export thousands of PubMed records with abstracts and MeSH terms to CSV/Excel in one run.
- **Research alerts:** schedule it daily with "only new papers" to get every new arXiv preprint in cs.CL or every new PubMed paper on your topic, without repeats.
- **AI agents and RAG:** feed abstracts and metadata to an LLM, a vector database or your agent through the API or Apify's MCP server.
- **Reference lists:** paste DOIs, PMIDs or arXiv links and get complete, consistent metadata for a bibliography.
- **Trend and competitor research:** track what a field, lab or keyword publishes over time (pharma, biotech, AI labs, investors).
- **Dataset building:** abstracts by category and date range for classification, topic modelling or benchmarks.

### How to use it

1. Type a **search query** (e.g. `large language models medicine`). Use `"quotes"` for exact phrases. Advanced syntax works too: arXiv prefixes `ti:`, `au:`, `abs:`, `cat:` and PubMed tags like `asthma[MeSH]`, `review[pt]`, `Smith J[au]`.
2. Choose **where to search**: arXiv + PubMed (default), arXiv only or PubMed only. Optionally add **arXiv categories** (`cs.CL`, `cs.AI`, `q-bio.GN`, `astro-ph`, ...) and a **date range**.
3. Or paste **paper IDs, DOIs or links** to look up specific papers.
4. Set **max papers per source** (default 50, up to 10,000) and click **Start**. Export the results as JSON, CSV, Excel or via the API.

### Pricing

You pay **$0.001 per paper** ($1 per 1,000) plus **$0.0005 per run start**. Duplicates, already-seen papers in monitoring mode, IDs that are not found and errors are **free**.

| Job | Papers | Price |
|---|---|---|
| Try it: 20 papers | 20 | **$0.02** |
| Daily alert, ~30 new papers a day for a month | ~900 | **~$0.92** |
| Literature review export | 1,000 | **$1.00** |
| Big PubMed export | 9,999 | **$10.00** |

Most arXiv and PubMed Actors in the Store charge $1.5-$3 per 1,000 results, some add a start fee of up to $0.09, and the all-in-one research MCP servers charge $0.02-$0.06 per search. Here one Actor covers both databases, merges duplicates and remembers what you have already seen.

Before each run it checks your **max cost per run** and never fetches more papers than that budget can pay for.

### Input example

```json
{
  "query": "retrieval augmented generation",
  "source": "arxiv",
  "arxivCategories": ["cs.CL", "cs.IR"],
  "dateFrom": "2026-10-01",
  "sortBy": "newest",
  "maxResults": 200,
  "onlyNew": true
}
```

Look up specific papers:

```json
{ "ids": ["2005.11401", "https://arxiv.org/abs/1706.03762", "33301246", "PMC7745181", "10.1038/s41586-020-2012-7"] }
```

### Output example

Real records from a run (abstracts and long lists shortened here). Every record has the **same fields in the same order**, whichever database it came from, so CSV and Excel exports line up.

```json
{
  "source": "pubmed",
  "id": "pubmed:33301246",
  "title": "Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine",
  "abstract": "BACKGROUND: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and the resulting coronavirus disease 2019 (Covid-19) have afflicted tens of millions of people in a worldwide pandemic. ...\nMETHODS: In an ongoing multinational, placebo-controlled, observer-blinded, pivotal efficacy trial, ...",
  "authors": ["Fernando P Polack", "Stephen J Thomas", "Nicholas Kitchin", "Judith Absalon", "..."],
  "authorCount": 30,
  "firstAuthor": "Fernando P Polack",
  "published": "2020-12-10",
  "updated": "2026-05-18",
  "year": 2020,
  "doi": "10.1056/nejmoa2034577",
  "arxivId": null,
  "version": null,
  "pmid": "33301246",
  "pmcid": "PMC7745181",
  "journal": "The New England journal of medicine",
  "journalAbbrev": "N Engl J Med",
  "journalRef": null,
  "issn": "1533-4406",
  "volume": "383",
  "issue": "27",
  "pages": "2603-2615",
  "primaryCategory": null,
  "categories": [],
  "meshTerms": ["Adolescent", "Adult", "Aged", "BNT162 Vaccine", "COVID-19", "COVID-19 Vaccines", "..."],
  "meshMajorTopics": ["COVID-19", "COVID-19 Vaccines", "SARS-CoV-2"],
  "keywords": [],
  "publicationTypes": ["Clinical Trial, Phase II", "Clinical Trial, Phase III", "Journal Article", "Multicenter Study", "Randomized Controlled Trial", "Research Support, Non-U.S. Gov't"],
  "language": "eng",
  "comment": null,
  "url": "https://pubmed.ncbi.nlm.nih.gov/33301246/",
  "pdfUrl": null,
  "doiUrl": "https://doi.org/10.1056/nejmoa2034577",
  "pubmedUrl": "https://pubmed.ncbi.nlm.nih.gov/33301246/",
  "pmcUrl": "https://pmc.ncbi.nlm.nih.gov/articles/PMC7745181/",
  "matchedBy": "id",
  "fetchedAt": "2026-10-21T07:20:52+00:00"
}
```

An arXiv record carries `arxivId`, `version`, `primaryCategory`, `categories`, the author `comment` (e.g. "Accepted at NeurIPS 2020"), `journalRef` and a direct `pdfUrl` such as `https://arxiv.org/pdf/2005.11401v4`. When the same paper is found in both databases, the kept record lists the other copy in `alsoFoundIn`.

The run summary (`OUTPUT` record) shows the total number of matches per database, so you know how much more there is. This one is from a real run with 10 newest papers per database on "large language models clinical" since 2026-09-01 plus 8 IDs (one unknown arXiv ID and one invalid input, both free):

```json
{"papersSaved": 25, "duplicatesRemoved": 0, "alreadySeenSkipped": 0, "notFound": 1, "errors": 1,
 "totalMatches": {"arxiv": 198, "pubmed": 1081}, "apiRequests": {"arxiv": 2, "pubmed": 6, "crossref": 1},
 "stateKey": "seen-platform-test-t1", "stoppedByMaxCharge": false}
```

### Monitoring: only new papers

Turn on **Only new papers**, sort by **Newest first** and schedule the Actor (Apify Schedules: daily or weekly). Each run remembers the papers it returned for that search (in a named key-value store in your account) and outputs only papers you have not seen. Already-seen papers are not charged. Use **Monitoring name** to keep several alerts apart, or to keep the memory when you edit the query. Connect the dataset to e-mail, Slack, Google Sheets, n8n or Make for an automatic alert.

### Use it from code, n8n or AI agents

Call it through the Apify API (`POST https://api.apify.com/v2/acts/manageable_gully~arxiv-pubmed-paper-search/run-sync-get-dataset-items`), the Apify Python/JavaScript clients, the n8n/Make/Zapier integrations, or as a tool for AI agents via the Apify MCP server.

### FAQ

**Where does the data come from?** From the official [arXiv API](https://info.arxiv.org/help/api/index.html), NCBI's official [E-utilities](https://www.ncbi.nlm.nih.gov/books/NBK25501/) for PubMed and the public [Crossref REST API](https://www.crossref.org/documentation/retrieve-metadata/rest-api/) (only for DOIs that are neither in PubMed nor on arXiv). It follows their usage rules: at most one arXiv request every 3 seconds and at most 3 PubMed requests per second (10 with your own NCBI API key). Thank you to arXiv for use of its open access interoperability.

**Does it download full texts?** No. It returns metadata, abstracts and links. arXiv papers come with a direct `pdfUrl`; PubMed papers with a free full text in PubMed Central have a `pmcUrl`. For the text of a PDF, run the PDF link through our [PDF & Image OCR Extractor](https://apify.com/manageable_gully/pdf-ocr-extractor). Respect the license of each paper.

**Why do I get at most 9,999 PubMed papers per search?** That is the E-utilities limit for one search. Split big searches into date ranges (e.g. one run per year) to get more.

**Is the abstract always included?** When the paper has one. Some PubMed records (letters, editorials, older papers) have no abstract. Turn **Include abstracts** off for smaller metadata-only datasets.

**Do I need an NCBI API key?** No. It is optional and only makes very large PubMed runs faster. You can get one for free in your NCBI account settings.

**Does it collect personal data?** It returns published bibliographic metadata: author names as printed on the paper and, only if you turn it on, affiliations with e-mail addresses removed. It does not collect contact details.

**Which arXiv categories exist?** See the [arXiv category taxonomy](https://arxiv.org/category_taxonomy). Examples: `cs.AI`, `cs.CL`, `cs.CV`, `cs.LG`, `stat.ML`, `q-bio.GN`, `physics.med-ph`, `econ.EM`. Use a top-level archive like `cs` or `math` to cover all its subcategories.

### More tools from WebIntel Tools

- [PDF & Image OCR to Text/Markdown](https://apify.com/manageable_gully/pdf-ocr-extractor): turn paper PDFs into text and tables
- [Web Page to Markdown for LLMs](https://apify.com/manageable_gully/page-to-markdown): clean Markdown and RAG chunks from any URL
- [RSS Feed Reader & Monitor](https://apify.com/manageable_gully/rss-feed-reader): journal and blog feeds, only new items
- [Website Screenshot & URL to PDF](https://apify.com/manageable_gully/url-to-pdf-screenshot)

# Actor input Schema

## `query` (type: `string`):

Keywords to search for, e.g. retrieval augmented generation. Use "quotes" for exact phrases. Advanced syntax is passed through: arXiv field prefixes (ti:, au:, abs:, cat:) and PubMed tags (asthma\[MeSH], review\[pt], Smith J\[au]).

## `source` (type: `string`):

arXiv is searched through the official arXiv API, PubMed through NCBI's official E-utilities.

## `arxivCategories` (type: `array`):

Optional. Limit arXiv results to these categories (any of them), e.g. cs.CL, cs.AI, cs.LG, q-bio.GN, astro-ph, stat.ML. A top-level archive like cs or math covers all its subcategories. Without a query, returns the newest papers in these categories.

## `dateFrom` (type: `string`):

Optional. Only papers submitted (arXiv) or published (PubMed) on or after this date.

## `dateTo` (type: `string`):

Optional. Only papers submitted/published on or before this date.

## `sortBy` (type: `string`):

Use 'Newest first' for monitoring and for 'only new papers' runs.

## `maxResults` (type: `integer`):

How many papers to fetch from each source (up to 10,000; PubMed allows 9,999 per search). Papers you look up by ID are always all returned.

## `ids` (type: `array`):

Optional. arXiv IDs (2005.11401, hep-th/9711200, arxiv.org links), PubMed IDs (33301246, pubmed links), PMCIDs (PMC7745181) or DOIs. DOIs are looked up on PubMed first, then on Crossref. Up to 10,000.

## `onlyNew` (type: `boolean`):

Remember which papers this search already returned and output only papers you have not seen before. Already-seen papers are free. Schedule the run daily/weekly with sort 'Newest first' to get a literature alert.

## `stateKey` (type: `string`):

Optional name for the 'only new' memory, e.g. crispr-weekly. By default it is derived from the search, so changing the query starts a fresh memory.

## `dedupe` (type: `boolean`):

Keep one record per paper when it appears in both arXiv and PubMed (same DOI or title). The kept record lists the other copy in alsoFoundIn. Duplicates are never charged.

## `includeAbstract` (type: `boolean`):

Turn off for smaller datasets with metadata only.

## `includeAffiliations` (type: `boolean`):

Add the authors' institutions as published in the paper record (e-mail addresses are removed).

## `crossrefFallback` (type: `boolean`):

For DOIs that are not in PubMed or arXiv (e.g. conference papers), fetch the metadata from the public Crossref API.

## `ncbiApiKey` (type: `string`):

Optional free key from your NCBI account. Raises PubMed's limit from 3 to 10 requests per second, which speeds up very large PubMed runs.

## Actor input object example

```json
{
  "query": "large language models medicine",
  "source": "both",
  "sortBy": "relevance",
  "maxResults": 50,
  "onlyNew": false,
  "dedupe": true,
  "includeAbstract": true,
  "includeAffiliations": false,
  "crossrefFallback": true
}
```

# Actor output Schema

## `results` (type: `string`):

One item per paper: title, abstract, authors, dates, DOI, journal, arXiv categories / MeSH terms and links (abstract page, PDF, PubMed, PMC).

## `summary` (type: `string`):

Papers saved, duplicates merged, already-seen papers skipped, total matches per source and API requests made.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "large language models medicine"
};

// Run the Actor and wait for it to finish
const run = await client.actor("manageable_gully/arxiv-pubmed-paper-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "large language models medicine" }

# Run the Actor and wait for it to finish
run = client.actor("manageable_gully/arxiv-pubmed-paper-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "large language models medicine"
}' |
apify call manageable_gully/arxiv-pubmed-paper-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,manageable_gully/arxiv-pubmed-paper-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rW5WUNCGTEkm5nSAI/builds/dtCgGTNeYJq5rO8IH/openapi.json