# PubMed Articles Scraper (`automation-lab/pubmed-article-search-scraper`) Actor

Search PubMed and export normalized article metadata and abstracts for literature reviews and research monitoring.

- **URL**: https://apify.com/automation-lab/pubmed-article-search-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Education
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.86 / 1,000 item extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## PubMed Articles Scraper

Search PubMed and export normalized **pubmed articles** for literature reviews, evidence mapping, research monitoring, and data pipelines.

The Actor uses the official NCBI E-utilities data surface. It turns PubMed queries into spreadsheet-ready records with PMID, title, structured abstract, authors, affiliations, journal, publication dates, publication types, MeSH terms, keywords, DOI, PMC ID, and canonical article URL.

No PubMed login, browser automation, or proxy is required.

### What does PubMed Articles Scraper do?

PubMed Articles Scraper accepts the same query language researchers use in the PubMed database.

For each query, it:

1. searches the official PubMed index;
2. pages through matching PMIDs;
3. downloads article metadata in batches;
4. normalizes nested XML into consistent JSON;
5. removes duplicate PMIDs across queries;
6. saves one article per default-dataset row.

You can search broadly, use PubMed field tags, restrict publication dates, require journals or publication types, and choose whether to include abstracts and MeSH terms.

### Who is this PubMed scraper for?

Use this Actor if you are:

- a researcher preparing a literature review;
- a medical librarian building a repeatable search export;
- a systematic-review team screening titles and abstracts;
- a biotech analyst monitoring a disease, drug target, or technique;
- a data engineer loading PubMed metadata into a warehouse;
- an evidence team comparing scheduled snapshots for new articles;
- a developer who needs normalized PubMed API output without writing an XML parser.

The Actor is designed for public bibliographic metadata. It does not retrieve paywalled full text.

### Why use this Actor instead of raw PubMed API calls?

NCBI E-utilities are powerful, but production workflows still need validation, pagination, throttling, retries, XML normalization, deduplication, storage, and scheduling.

This Actor provides those pieces in one run:

- typed dataset rows instead of raw XML;
- structured authors and flattened author names;
- author-level and article-level affiliations;
- DOI and PMC identifiers resolved from PubMed identifiers;
- query and rank context on every row;
- global `maxItems` enforcement across several queries;
- safe NCBI request-rate handling;
- transient-error retries with bounded backoff;
- direct Apify scheduling, API, webhook, and integration support.

### What PubMed article data can I extract?

| Field | Meaning |
| --- | --- |
| `pmid` | Stable PubMed identifier |
| `title` | Article title, including text from inline scientific markup |
| `abstract` | Structured abstract sections joined into readable text |
| `authors` | Names, initials, affiliations, collective names, and ORCID when indexed |
| `authorNames` | Flat list of author display names |
| `affiliations` | Unique affiliations aggregated across authors |
| `journal` | Full journal title |
| `journalAbbreviation` | NLM or ISO abbreviation |
| `journalIssn` | ISSN when available |
| `publicationDate` | Normalized issue publication date or source date text |
| `electronicPublicationDate` | Separate electronic publication date when available |
| `articleTypes` | Journal Article, Review, Clinical Trial, Meta-Analysis, and other PubMed types |
| `meshTerms` | Assigned Medical Subject Headings |
| `keywords` | Indexed author or article keywords |
| `languages` | PubMed language codes |
| `doi` | Digital Object Identifier |
| `pmcId` | PubMed Central identifier when available |
| `articleIds` | All identifiers and their namespaces |
| `url` | Canonical `pubmed.ncbi.nlm.nih.gov` article URL |
| `query` | Input query that produced the row |
| `rank` | Rank within that query result |
| `totalResultsForQuery` | Total count reported by PubMed |
| `retrievedAt` | Retrieval timestamp |

Fields remain absent or empty when PubMed does not index that information. An article is not rejected merely because it has no abstract or DOI.

### How to search PubMed articles

#### 1. Open the Actor

Open PubMed Articles Scraper in Apify Console and select **Try for free**.

#### 2. Enter a query

A simple input is:

```json
{
  "queries": ["CRISPR cancer"],
  "maxItems": 20,
  "sort": "pub_date",
  "includeAbstract": true,
  "includeMeshTerms": true
}
```

#### 3. Run the Actor

The log reports each PubMed result page and the number of unique records saved.

#### 4. Export the dataset

Download results as JSON, CSV, Excel, XML, RSS, or JSONL, or consume the dataset through the Apify API.

### PubMed query examples

The `queries` field supports standard PubMed search syntax.

Broad topic search:

```text
CRISPR cancer
```

Medical Subject Heading:

```text
Alzheimer Disease[MeSH Terms]
```

Boolean evidence query:

```text
(diabetes OR hyperglycemia) AND telemedicine
```

Publication-type query:

```text
immunotherapy AND Meta-Analysis[Publication Type]
```

Author query:

```text
Smith J[Author] AND cardiology
```

You can put up to 20 queries in one run. If the same PMID matches several queries, the Actor saves it once, using the first query that produced it.

### Input parameters

| Input | Type | Default | Description |
| --- | --- | --- | --- |
| `queries` | string array | `CRISPR cancer` | One to 20 PubMed queries |
| `maxItems` | integer | `100` | Maximum unique records across all queries, from 1 to 10,000 |
| `sort` | string | `relevance` | `relevance` or newest-first `pub_date` |
| `minDate` | string | none | Inclusive publication date in `YYYY/MM/DD` format |
| `maxDate` | string | none | Inclusive publication date in `YYYY/MM/DD` format |
| `articleTypes` | string array | empty | Exact PubMed publication types to require |
| `journals` | string array | empty | Exact journal names to require |
| `includeAbstract` | boolean | `true` | Include abstract text when indexed |
| `includeMeshTerms` | boolean | `true` | Include assigned MeSH terms |
| `apiKey` | secret string | none | Optional personal NCBI API key for a higher request rate |
| `email` | string | none | Optional contact email sent to NCBI |

`minDate` and `maxDate` apply to PubMed publication dates. They do not filter the Actor's retrieval timestamp.

Publication-type and journal inputs are translated into standard PubMed field filters. You can also write advanced field tags directly in each query.

### Example output

A current run produces records shaped like this:

```json
{
  "pmid": "42378068",
  "title": "Is Fusobacterium nucleatum the key mediator between oral infections and systemic diseases? Mechanistic insights and therapeutic implications.",
  "abstract": "Fusobacterium nucleatum has emerged as a pathobiont...",
  "authors": [
    {
      "name": "Sample Author",
      "lastName": "Author",
      "foreName": "Sample",
      "initials": "S",
      "affiliations": ["Department of Medicine, Sample University"]
    }
  ],
  "authorNames": ["Sample Author"],
  "affiliations": ["Department of Medicine, Sample University"],
  "journal": "Gut microbes",
  "publicationDate": "2026-12-31",
  "articleTypes": ["Journal Article", "Review"],
  "meshTerms": ["Fusobacterium nucleatum"],
  "keywords": [],
  "languages": ["eng"],
  "doi": "10.1080/19490976.2026.2694819",
  "articleIds": [
    { "type": "pubmed", "value": "42378068" },
    { "type": "doi", "value": "10.1080/19490976.2026.2694819" }
  ],
  "url": "https://pubmed.ncbi.nlm.nih.gov/42378068/",
  "query": "CRISPR cancer",
  "queryIndex": 1,
  "rank": 1,
  "totalResultsForQuery": 16735,
  "source": "PubMed",
  "retrievedAt": "2026-08-28T06:50:00.000Z"
}
```

The example anonymizes author details. Actual dataset rows preserve public author and affiliation metadata supplied by PubMed.

### How much does it cost to export PubMed articles?

PubMed Articles Scraper uses pay-per-event pricing:

- **$0.001** when a valid run starts;
- a tiered amount for each unique PubMed article saved;
- **$0.003096 per article on the BRONZE tier** before higher-volume discounts.

At BRONZE pricing, approximate Actor charges are:

| Saved articles | Approximate charge |
| ---: | ---: |
| 10 | $0.03196 |
| 100 | $0.31060 |
| 1,000 | $3.09700 |

The final amount depends on your Apify pricing tier and the number of records actually saved. Empty searches incur only the run-start event. Duplicate PMIDs are not charged twice.

NCBI does not require a paid API subscription for this Actor. Apify platform compute is included through the Actor's event pricing according to your plan.

### Literature review workflow

A practical repeatable workflow is:

1. define and test a PubMed query;
2. run the Actor with abstracts and MeSH terms enabled;
3. export the default dataset to CSV or your review tool;
4. save the query input as an Apify Task;
5. schedule the Task weekly or monthly;
6. compare PMID values with the previous dataset;
7. send newly observed articles to screening or alerting automation.

The Actor returns snapshots. It does not maintain a cross-run history or send alerts by itself. Use Apify schedules, webhooks, dataset APIs, or your own database for those steps.

### Spreadsheet and data-pipeline integrations

You can connect the dataset to:

- Google Sheets for title and abstract screening;
- Microsoft Excel through a CSV or XLSX export;
- Make or Zapier through an Actor-run webhook;
- Python or R analysis notebooks;
- a warehouse keyed by `pmid`;
- a vector database for permitted abstract search;
- a review-management system that accepts tabular imports.

Use `pmid` as the stable deduplication key. DOI is useful but is not present on every PubMed record.

### Run PubMed Articles Scraper with the API

Replace `YOUR_TOKEN` with an Apify API token.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~pubmed-article-search-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["Alzheimer Disease[MeSH Terms]"],
    "minDate": "2025/01/01",
    "maxItems": 20,
    "sort": "pub_date"
  }'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/pubmed-article-search-scraper').call({
  queries: ['CRISPR cancer'],
  maxItems: 100,
  sort: 'pub_date',
  includeAbstract: true,
  includeMeshTerms: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient(token="YOUR_TOKEN")
run = client.actor("automation-lab/pubmed-article-search-scraper").call(
    run_input={
        "queries": ["immunotherapy AND Meta-Analysis[Publication Type]"],
        "maxItems": 100,
        "sort": "pub_date",
    }
)

items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

For asynchronous jobs, start a normal Actor run and poll its status or receive an Apify webhook instead of using the synchronous endpoint.

### Use the Actor through MCP

Add the Apify MCP server to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"
```

#### Claude Desktop

Add this server object to Claude Desktop's MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"
    }
  }
}
```

#### Cursor

Open **Settings → MCP**, add a remote HTTP server, and use:

```text
https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper
```

#### VS Code

Add an HTTP MCP server in your VS Code MCP configuration with the same endpoint:

```json
{
  "servers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"
    }
  }
}
```

Example prompts:

- "Search PubMed for the 25 newest CRISPR delivery articles and return titles, abstracts, DOI, and MeSH terms."
- "Find randomized controlled trials in Lancet matching medicine and save the normalized dataset."
- "Run my Alzheimer Disease MeSH monitoring query from 2025 onward and summarize the new article topics."

Your MCP client still needs access to your Apify account and token according to its setup.

### Limits and responsible NCBI usage

The Actor intentionally uses the official anonymous NCBI E-utilities route.

Important limits:

- a run accepts at most 20 queries and 10,000 unique articles;
- anonymous requests are throttled to three per second;
- a supplied NCBI API key enables a bounded rate below ten per second;
- larger result sets take longer because article metadata is fetched in batches;
- PubMed can return estimated or changing counts while its index updates;
- abstracts, MeSH terms, DOI, ORCID, and affiliations are not present on every article;
- full text is outside this Actor's scope;
- PubMed Central IDs indicate availability in PMC but do not make every use unrestricted.

The Actor retries temporary network errors, HTTP 429 responses, and NCBI server errors. It fails the run after bounded retries rather than silently returning a partial success as complete.

### Legal and ethical use

PubMed bibliographic records are public, but article abstracts, full text, and linked publisher content may have separate rights and licenses.

You are responsible for:

- following NCBI policies and attribution guidance;
- using an NCBI API key only if you are authorized to use it;
- respecting publisher copyright and database rights;
- avoiding attempts to retrieve paywalled full text through this Actor;
- handling public author and affiliation data for a legitimate research purpose;
- complying with privacy, research, and institutional requirements that apply to your project.

This Actor does not bypass access controls and does not log into PubMed.

### Troubleshooting

#### Why did the run return no articles?

Open the same query in PubMed and check field names, quotation marks, date limits, journal names, and publication types. Several restrictive filters are combined, so one misspelled journal can reduce the result to zero.

#### Why is an abstract or DOI missing?

PubMed does not index those fields for every record. The Actor preserves the article with an absent field rather than inventing data.

#### Why did NCBI return HTTP 429?

The Actor already throttles requests and retries transient 429 responses. Repeated failures can indicate unusual upstream load. Retry later, reduce concurrent runs, or provide your own authorized NCBI API key.

#### Why is `publicationDate` not a full ISO date?

Some records only contain a year, month, season, or date range. The Actor normalizes precise components when present and preserves PubMed's textual date when that is the only source value.

#### Can I request more than 10,000 records?

Not in one run. Split the research question into defensible date ranges or topic queries, then deduplicate by PMID in your destination system.

### Frequently asked questions

#### Does this scrape PubMed Central full text?

No. It searches PubMed article records and returns metadata and abstracts indexed by PubMed. A `pmcId` is included when PubMed links the record to PubMed Central.

#### Does it support PubMed advanced search syntax?

Yes. Put supported PubMed Boolean expressions and field tags directly in `queries`.

#### Can I monitor newly indexed articles?

Yes. Save the input as a scheduled Apify Task, sort by publication date, use a date boundary, and compare PMIDs with your previous dataset. The Actor itself returns each run's snapshot and does not maintain alerts.

#### Are duplicate articles charged twice?

No. Within a run, repeated PMIDs across queries are saved and charged once.

#### Is an NCBI API key required?

No. It is optional. Anonymous runs use the official lower request limit.

#### Can I export to CSV or Excel?

Yes. Open the run's default dataset and select CSV or Excel, or request the corresponding dataset API format.

### Related Actors

This Actor currently operates as a standalone PubMed literature-data product. The automation-lab portfolio does not yet have another active public Actor with the same biomedical article workflow, so this README does not recommend an unrelated product merely to create a cross-link.

For adjacent workflows, combine the dataset with Apify's schedules, webhooks, storage integrations, and your own screening or analysis tools.

### Support

If a query behaves differently from PubMed, include:

- the Actor run URL;
- the exact input with secrets removed;
- the expected PubMed behavior;
- one PMID that demonstrates the discrepancy, if available.

Do not include your NCBI API key in a support message or dataset.

# Actor input Schema

## `queries` (type: `array`):

One or more PubMed queries. Standard PubMed field tags and Boolean operators are supported, for example Alzheimer disease\[MeSH Terms] AND review\[Publication Type].

## `maxItems` (type: `integer`):

Maximum number of unique article records saved across all queries. Duplicate PMIDs found by multiple queries count once.

## `sort` (type: `string`):

Order returned by PubMed: best match relevance or newest publication date first.

## `minDate` (type: `string`):

Optional inclusive publication-date boundary in YYYY/MM/DD format.

## `maxDate` (type: `string`):

Optional inclusive publication-date boundary in YYYY/MM/DD format.

## `articleTypes` (type: `array`):

Optional exact PubMed publication types, such as Review, Clinical Trial, or Meta-Analysis. Values are combined with AND filters.

## `journals` (type: `array`):

Optional journal names to require, such as The Lancet or Nature Medicine. Values are combined with AND filters.

## `includeAbstract` (type: `boolean`):

Include abstract text when PubMed provides it. Disabling this reduces dataset size but does not change request count.

## `includeMeshTerms` (type: `boolean`):

Include Medical Subject Headings assigned to each article when available.

## `apiKey` (type: `string`):

Optional personal NCBI API key. It permits a higher request rate but is not required for normal runs.

## `email` (type: `string`):

Optional contact email sent to NCBI E-utilities as recommended for larger automated requests.

## Actor input object example

```json
{
  "queries": [
    "CRISPR cancer"
  ],
  "maxItems": 20,
  "sort": "relevance",
  "articleTypes": [],
  "journals": [],
  "includeAbstract": true,
  "includeMeshTerms": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing normalized article metadata and abstracts.

## `overview` (type: `string`):

Table view of the most useful article fields.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "CRISPR cancer"
    ],
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/pubmed-article-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["CRISPR cancer"],
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/pubmed-article-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "CRISPR cancer"
  ],
  "maxItems": 20
}' |
apify call automation-lab/pubmed-article-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/pubmed-article-search-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/irbMXh69VuTrrIYNZ/builds/TQF83SdDDvgOe9A2w/openapi.json