# PubMed Scraper: Biomedical Literature & Abstracts (`arman-bd/pubmed-articles-scraper`) Actor

Search 35M+ PubMed citations and get complete records: title, structured abstract, authors and affiliations, journal, MeSH terms, keywords, DOI and PMC ID. Straight from NCBI E-utilities, no key required.

- **URL**: https://apify.com/arman-bd/pubmed-articles-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Business, Developer tools, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.56 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## PubMed Scraper: Biomedical Literature & Abstracts

![PubMed Scraper: Full biomedical citations by PubMed query, structured abstracts, authors, affiliations, MeSH terms, DOI and PMC ID](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/pubmed-articles-scraper.jpg)

**PubMed Scraper** searches the 35M+ citations in PubMed and returns one flat record per article, title, full abstract with its section labels intact, authors and their affiliations, journal placement, MeSH terms, keywords, DOI and PMC ID.

It reads the data the NIH publishes specifically for programmatic access: **no browser, no proxies, no login, no scraping of HTML pages.** An optional free NCBI key triples throughput; without one the Actor paces itself automatically.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/pubmed-articles-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/pubmed-articles-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `pmid`, `doi`, `pmcId`, `url` | PubMed ID, publisher DOI, PubMed Central ID and the PubMed page |
| `title` | Article title, inline `<i>`/`<sup>` markup stripped |
| `abstract` | Full abstract. Structured abstracts keep their headings, `BACKGROUND: … METHODS: …` |
| `authors` | Author names in order; consortium authors appear under their collective name |
| `affiliations` | Every distinct affiliation string on the record |
| `journal`, `journalAbbrev`, `issn` | Full journal title, its ISO abbreviation and ISSN |
| `volume`, `issue`, `pages` | Issue placement |
| `publicationDate`, `publicationYear` | Most precise date the record supports, `2020-12-31`, `2026-12` or `1929` |
| `meshTerms` | NLM Medical Subject Headings assigned by indexers |
| `keywords` | Author-supplied keywords |
| `publicationTypes` | `Journal Article`, `Review`, `Randomized Controlled Trial`, `Published Erratum`… |
| `language`, `languages`, `country` | Primary publication language, every language the record lists, and the journal's country of origin |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, every query that failed or matched nothing, and the PMIDs that were unusable or resolved to no record.

### Common use cases

- **Systematic reviews.** One query per PICO arm, filtered to `Review` and `Meta-Analysis`, exported to CSV for screening.
- **Biomedical RAG corpora.** `title` + `abstract` + `meshTerms` is a chunk with its own controlled vocabulary attached.
- **Institutional output tracking.** Search `"Karolinska Institutet"[Affiliation]` and count by year.
- **Pharmacovigilance and horizon scanning.** Schedule a daily run sorted by publication date and diff on `pmid`.
- **DOI reconciliation.** Feed in PMIDs and get DOIs and PMC IDs back for joining onto other sources.

### Quick start

One topic, the 100 best matches:

```json
{
 "searchQueries": ["CRISPR gene editing"]
}
```

Recent English-language reviews and meta-analyses:

```json
{
 "searchQueries": ["long covid[MeSH Terms]", "post-acute sequelae SARS-CoV-2"],
 "articleTypes": ["Review", "Meta-Analysis"],
 "language": "eng",
 "fromDate": "2024-01-01",
 "sortBy": "pub_date",
 "maxResults": 500,
 "email": "you@example.com"
}
```

Enrich a list of PMIDs you already have, metadata only:

```json
{
 "pmids": ["33301246", "32109013"],
 "includeAbstract": false
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `searchQueries` | array | `[]` | PubMed query syntax. Field tags work: `[Title]`, `[Author]`, `[Journal]`, `[MeSH Terms]`, `[Affiliation]`. Combine with `AND` / `OR` / `NOT`. |
| `pmids` | array | `[]` | Specific articles. Bare PMID, `PMID: 33301246`, or a pubmed.ncbi.nlm.nih.gov URL. |
| `fromDate` / `toDate` | string | - | Publication date bounds as `YYYY-MM-DD`, `YYYY-MM` or `YYYY`. Applied as PubMed's own `pdat` filter. A partial `toDate` closes on the real last day of the period, so `2024-02` ends on the 29th in 2024 and the 28th in 2023. |
| `articleTypes` | array | `[]` | Publication types, ORed together and ANDed onto every query. |
| `language` | string | - | Three-letter code, `eng`, `fre`, `ger`, `spa`. |
| `sortBy` | string | `relevance` | `relevance` (Best Match), `pub_date`, `Author` or `JournalName`. |
| `includeAbstract` | boolean | `true` | Include the full abstract. Off makes the dataset several times smaller. |
| `maxResults` | integer | `100` | Articles saved **per source** — each search query is a source, and the whole `pmids` list is one more. Hard ceiling 9,999, see Limits. |
| `email` | string | - | Passed to NCBI so it can contact you rather than block you. Recommended. |
| `apiKey` | string | - | Free NCBI key. Raises the rate limit from 3 to 10 requests/second. |

At least one of `searchQueries` or `pmids` is required, PubMed has no "return everything" mode. Both can be used in the same run; results are de-duplicated by `pmid`.

### Output example

A real record from the run above:

```json
{
 "pmid": "42522380",
 "doi": "10.1080/21691401.2026.2702869",
 "pmcId": null,
 "title": "Recent advances of CRISPR-based gene editing technologies and delivery strategies.",
 "abstract": "CRISPR technology is a powerful tool for gene editing, in which the efficient delivery of living target cells allows it to show great clinical potential. …",
 "authors": ["Fanyu Xue", "Zixuan Xin", "Gaojie Wang", "Jiayin Xing", "Hengxin Han", "Xiangxiang Zhao", "Ningning Song"],
 "affiliations": ["College of Life Science and Technology, Shandong Second Medical University, Weifang, China."],
 "journal": "Artificial cells, nanomedicine, and biotechnology",
 "journalAbbrev": "Artif Cells Nanomed Biotechnol",
 "issn": "2169-141X",
 "volume": "54",
 "issue": "1",
 "pages": "415-431",
 "publicationDate": "2026-12",
 "publicationYear": 2026,
 "meshTerms": ["Humans", "Gene Editing", "Animals", "CRISPR-Cas Systems", "Gene Transfer Techniques"],
 "keywords": ["Adeno-associated viral vectors", "CRISPR-Cas", "LNP delivery systems", "delivery system"],
 "publicationTypes": ["Journal Article", "Review"],
 "language": "eng",
 "languages": ["eng"],
 "country": "England",
 "url": "https://pubmed.ncbi.nlm.nih.gov/42522380/",
 "scrapedAt": "2026-08-06T12:15:43.691Z"
}
```

`meshTerms` is empty for very recent records, NLM indexers assign headings weeks to months after publication, so a paper from last week has `keywords` but no MeSH. `pmcId` is only present when a free full text exists in PubMed Central.

### Writing a PubMed query

The `searchQueries` field is passed to PubMed untouched, so anything that works in the search box works here:

| Goal | Query |
|---|---|
| Word in the title | `CRISPR[Title]` |
| Specific author | `Doudna JA[Author]` |
| Specific journal | `Nature[Journal]` |
| Controlled vocabulary | `"gene editing"[MeSH Terms]` |
| One institution | `"Karolinska Institutet"[Affiliation]` |
| Combined | `(CRISPR[Title] OR Cas9[Title]) AND therapy NOT review[Publication Type]` |

Plain keywords work too, PubMed's term mapping expands `cancer` into a large synonym set on your behalf. `RUN_SUMMARY.filters` records exactly what was sent.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~pubmed-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "searchQueries": ["CRISPR gene editing"],
 "articleTypes": ["Review"],
 "fromDate": "2025-01-01",
 "maxResults": 50
 }'
```

### Limits and behaviour

- **Two steps per query.** esearch returns PMIDs, then efetch pulls the records 200 at a time. A 1,000-article run is one esearch plus five efetch calls, so cost per article falls sharply as runs get bigger.
- **9,999 results per query is PubMed's ceiling.** esearch refuses `retstart` above 9998 and clamps `retmax` to 9999. Slice a broader sweep with `fromDate` / `toDate` rather than raising the cap.
- **Rate limits are respected.** 3 requests/second without a key, 10 with one. The Actor paces itself to stay under whichever applies, and NCBI's `Retry-After` is honoured on the rare 429.
- **XML is parsed without a dependency.** efetch answers XML, read here by a small purpose-built extractor. Presentational markup is stripped before entities are decoded, so `α` arrives as α and an escaped `&lt;i&gt;` stays visible text.
- **Books are handled too.** PubMed indexes NCBI Bookshelf entries such as GeneReviews as `PubmedBookArticle` records with a different structure. They are parsed rather than dropped, but carry fewer fields, no volume, issue or MeSH.
- **Unknown PMIDs are named, not swallowed.** efetch returns an empty set rather than an error for an ID that does not exist, so the run diffs what came back against what you asked for: entries that hold no ID at all land in `RUN_SUMMARY.invalidPmids`, IDs that resolve to no record in `RUN_SUMMARY.unresolvedPmids`.
- **A query that matches nothing is a recorded outcome.** A zero-result search is not an error on the wire, so the run reads PubMed's own warning list and writes the reason into `RUN_SUMMARY.failures` with `kind: "empty"` instead of finishing quietly with no rows.
- **A failing query never aborts the run.** It is logged and recorded in `RUN_SUMMARY.failures` with `kind: "error"`; the rest continue. The run only errors out if *every* source fails outright.
- **Public data only.** PubMed citations are public records. No authentication beyond the optional rate-limit key, no personal data, no access-control bypass.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need an NCBI account?** Not to run it. The API key is optional and only affects speed.

**Why should I fill in `email`?** NCBI asks automated clients to identify themselves and contacts the address before blocking anything that looks abusive. It is never used by this Actor for anything else.

**What happens if a query fails?** It is reported in `RUN_SUMMARY.failures` and the run continues with the rest.

**Can I schedule it?** Yes. Pair `fromDate` with `sortBy: pub_date` and diff on `pmid` for a "what's new this week" feed.

**Does it fetch full text?** No, the abstract only. When `pmcId` is present the full text is free at `ncbi.nlm.nih.gov/pmc/articles/{pmcId}/`.

**Why is `abstract` null?** Some records have none, editorials, errata, letters and many older papers. Nothing is invented.

**Can I search MEDLINE only, or exclude preprints?** Yes, with normal PubMed syntax: append `NOT preprint[Publication Type]` or `AND medline[Subset]` to your query.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `searchQueries` (type: `array`):

PubMed query syntax, one query per entry. exactly what you would type into pubmed.ncbi.nlm.nih.gov. Field tags work: 'CRISPR\[Title]', 'Doudna JA\[Author]', 'Nature\[Journal]', '"gene editing"\[MeSH Terms]'. Combine with AND / OR / NOT. Plain keywords work too and are expanded by PubMed's own term mapping.

## `pmids` (type: `array`):

Fetch specific articles instead of, or alongside, searching. Accepts a bare PMID (33301246), 'PMID: 33301246', or a pubmed.ncbi.nlm.nih.gov URL. Entries that hold no ID are listed in RUN\_SUMMARY.invalidPmids, and IDs that resolve to no record in RUN\_SUMMARY.unresolvedPmids. 'Max results' caps this list too.

## `fromDate` (type: `string`):

Earliest publication date, as YYYY-MM-DD, YYYY-MM or YYYY. Applied as PubMed's own 'pdat' date filter, not as text in the query. A month or a bare year resolves to its first day. Leave empty for no lower bound.

## `toDate` (type: `string`):

Latest publication date, same formats as 'Published from'. A month resolves to that month's real last day (28, 29, 30 or 31) and a bare year to 31 December, so 2024-02 and 2024 both mean the whole period. A date that does not exist is rejected. Leave empty for no upper bound.

## `articleTypes` (type: `array`):

Keep only these PubMed publication types. They are ORed together and ANDed onto every query, so picking Review and Meta-Analysis returns records that are either. Empty means no type filter.

## `language` (type: `string`):

Restrict to one publication language using PubMed's three-letter code. 'eng', 'fre', 'ger', 'spa', 'chi', 'jpn'. Empty means every language. Note this is the language the paper was published in; abstracts of non-English papers are usually still in English.

## `sortBy` (type: `string`):

How PubMed orders matches before the result cap is applied. 'relevance' is PubMed's Best Match ranking; 'pub\_date' puts the newest first and is what you want for a scheduled 'what is new' run.

## `includeAbstract` (type: `boolean`):

Include the full abstract on every record. Structured abstracts keep their section labels (BACKGROUND, METHODS, RESULTS…). Turning this off makes the dataset several times smaller. useful for change-detection sweeps.

## `maxResults` (type: `integer`):

Cap on articles saved, and charged for, per source: each search query counts as one source and the whole PMID list as another. PubMed's esearch cannot return more than 9,999 records for one query no matter what you ask for, so that is also the ceiling here. Split a broader sweep by date range instead.

## `email` (type: `string`):

NCBI asks every automated client to identify itself and emails you before blocking anything that misbehaves. Nothing is sent to this address by the Actor. it is passed straight to E-utilities. Recommended for scheduled or heavy runs.

## `apiKey` (type: `string`):

Optional. Raises the rate limit from 3 to 10 requests per second, which roughly triples throughput on large runs. Free from your NCBI account settings page (Account settings → API Key Management).

## Actor input object example

```json
{
  "searchQueries": [
    "CRISPR[Title] AND therapy",
    "long covid[MeSH Terms]"
  ],
  "pmids": [
    "33301246",
    "32109013"
  ],
  "fromDate": "2024-01-01",
  "toDate": "2026-12-31",
  "articleTypes": [
    "Review",
    "Meta-Analysis"
  ],
  "language": "eng",
  "sortBy": "relevance",
  "includeAbstract": true,
  "maxResults": 100,
  "email": "you@example.com",
  "apiKey": "your-ncbi-api-key"
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "CRISPR gene editing"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/pubmed-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQueries": ["CRISPR gene editing"] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/pubmed-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "CRISPR gene editing"
  ]
}' |
apify call arman-bd/pubmed-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/pubmed-articles-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bLgaiKDEwmsOiDSvh/builds/N2Os3LbB1Pd2gZ6jH/openapi.json