PubMed Articles Scraper

Search PubMed and export normalized pubmed articles for literature reviews, evidence mapping, research monitoring, and data pipelines.

The Actor uses the official NCBI E-utilities data surface. It turns PubMed queries into spreadsheet-ready records with PMID, title, structured abstract, authors, affiliations, journal, publication dates, publication types, MeSH terms, keywords, DOI, PMC ID, and canonical article URL.

No PubMed login, browser automation, or proxy is required.

What does PubMed Articles Scraper do?

PubMed Articles Scraper accepts the same query language researchers use in the PubMed database.

For each query, it:

1. searches the official PubMed index;
2. pages through matching PMIDs;
3. downloads article metadata in batches;
4. normalizes nested XML into consistent JSON;
5. removes duplicate PMIDs across queries;
6. saves one article per default-dataset row.

You can search broadly, use PubMed field tags, restrict publication dates, require journals or publication types, and choose whether to include abstracts and MeSH terms.

Who is this PubMed scraper for?

Use this Actor if you are:

• a researcher preparing a literature review;
• a medical librarian building a repeatable search export;
• a systematic-review team screening titles and abstracts;
• a biotech analyst monitoring a disease, drug target, or technique;
• a data engineer loading PubMed metadata into a warehouse;
• an evidence team comparing scheduled snapshots for new articles;
• a developer who needs normalized PubMed API output without writing an XML parser.

The Actor is designed for public bibliographic metadata. It does not retrieve paywalled full text.

Why use this Actor instead of raw PubMed API calls?

NCBI E-utilities are powerful, but production workflows still need validation, pagination, throttling, retries, XML normalization, deduplication, storage, and scheduling.

This Actor provides those pieces in one run:

• typed dataset rows instead of raw XML;
• structured authors and flattened author names;
• author-level and article-level affiliations;
• DOI and PMC identifiers resolved from PubMed identifiers;
• query and rank context on every row;
• global maxItems enforcement across several queries;
• safe NCBI request-rate handling;
• transient-error retries with bounded backoff;
• direct Apify scheduling, API, webhook, and integration support.

What PubMed article data can I extract?

Field	Meaning
pmid	Stable PubMed identifier
title	Article title, including text from inline scientific markup
abstract	Structured abstract sections joined into readable text
authors	Names, initials, affiliations, collective names, and ORCID when indexed
authorNames	Flat list of author display names
affiliations	Unique affiliations aggregated across authors
journal	Full journal title
journalAbbreviation	NLM or ISO abbreviation
journalIssn	ISSN when available
publicationDate	Normalized issue publication date or source date text
electronicPublicationDate	Separate electronic publication date when available
articleTypes	Journal Article, Review, Clinical Trial, Meta-Analysis, and other PubMed types
meshTerms	Assigned Medical Subject Headings
keywords	Indexed author or article keywords
languages	PubMed language codes
doi	Digital Object Identifier
pmcId	PubMed Central identifier when available
articleIds	All identifiers and their namespaces
url	Canonical pubmed.ncbi.nlm.nih.gov article URL
query	Input query that produced the row
rank	Rank within that query result
totalResultsForQuery	Total count reported by PubMed
retrievedAt	Retrieval timestamp

Fields remain absent or empty when PubMed does not index that information. An article is not rejected merely because it has no abstract or DOI.

How to search PubMed articles

1. Open the Actor

Open PubMed Articles Scraper in Apify Console and select Try for free.

2. Enter a query

A simple input is:

{
  "queries": ["CRISPR cancer"],
  "maxItems": 20,
  "sort": "pub_date",
  "includeAbstract": true,
  "includeMeshTerms": true
}

3. Run the Actor

The log reports each PubMed result page and the number of unique records saved.

4. Export the dataset

Download results as JSON, CSV, Excel, XML, RSS, or JSONL, or consume the dataset through the Apify API.

PubMed query examples

The queries field supports standard PubMed search syntax.

Broad topic search:

CRISPR cancer

Medical Subject Heading:

Alzheimer Disease[MeSH Terms]

Boolean evidence query:

(diabetes OR hyperglycemia) AND telemedicine

Publication-type query:

immunotherapy AND Meta-Analysis[Publication Type]

Author query:

Smith J[Author] AND cardiology

You can put up to 20 queries in one run. If the same PMID matches several queries, the Actor saves it once, using the first query that produced it.

Input parameters

Input	Type	Default	Description
queries	string array	CRISPR cancer	One to 20 PubMed queries
maxItems	integer	100	Maximum unique records across all queries, from 1 to 10,000
sort	string	relevance	relevance or newest-first pub_date
minDate	string	none	Inclusive publication date in YYYY/MM/DD format
maxDate	string	none	Inclusive publication date in YYYY/MM/DD format
articleTypes	string array	empty	Exact PubMed publication types to require
journals	string array	empty	Exact journal names to require
includeAbstract	boolean	true	Include abstract text when indexed
includeMeshTerms	boolean	true	Include assigned MeSH terms
apiKey	secret string	none	Optional personal NCBI API key for a higher request rate
email	string	none	Optional contact email sent to NCBI

minDate and maxDate apply to PubMed publication dates. They do not filter the Actor's retrieval timestamp.

Publication-type and journal inputs are translated into standard PubMed field filters. You can also write advanced field tags directly in each query.

Example output

A current run produces records shaped like this:

{
  "pmid": "42378068",
  "title": "Is Fusobacterium nucleatum the key mediator between oral infections and systemic diseases? Mechanistic insights and therapeutic implications.",
  "abstract": "Fusobacterium nucleatum has emerged as a pathobiont...",
  "authors": [
    {
      "name": "Sample Author",
      "lastName": "Author",
      "foreName": "Sample",
      "initials": "S",
      "affiliations": ["Department of Medicine, Sample University"]
    }
  ],
  "authorNames": ["Sample Author"],
  "affiliations": ["Department of Medicine, Sample University"],
  "journal": "Gut microbes",
  "publicationDate": "2026-12-31",
  "articleTypes": ["Journal Article", "Review"],
  "meshTerms": ["Fusobacterium nucleatum"],
  "keywords": [],
  "languages": ["eng"],
  "doi": "10.1080/19490976.2026.2694819",
  "articleIds": [
    { "type": "pubmed", "value": "42378068" },
    { "type": "doi", "value": "10.1080/19490976.2026.2694819" }
  ],
  "url": "https://pubmed.ncbi.nlm.nih.gov/42378068/",
  "query": "CRISPR cancer",
  "queryIndex": 1,
  "rank": 1,
  "totalResultsForQuery": 16735,
  "source": "PubMed",
  "retrievedAt": "2026-08-28T06:50:00.000Z"
}

The example anonymizes author details. Actual dataset rows preserve public author and affiliation metadata supplied by PubMed.

How much does it cost to export PubMed articles?

PubMed Articles Scraper uses pay-per-event pricing:

• $0.001 when a valid run starts;
• a tiered amount for each unique PubMed article saved;
• $0.003096 per article on the BRONZE tier before higher-volume discounts.

At BRONZE pricing, approximate Actor charges are:

Saved articles	Approximate charge
10	$0.03196
100	$0.31060
1,000	$3.09700

The final amount depends on your Apify pricing tier and the number of records actually saved. Empty searches incur only the run-start event. Duplicate PMIDs are not charged twice.

NCBI does not require a paid API subscription for this Actor. Apify platform compute is included through the Actor's event pricing according to your plan.

Literature review workflow

A practical repeatable workflow is:

1. define and test a PubMed query;
2. run the Actor with abstracts and MeSH terms enabled;
3. export the default dataset to CSV or your review tool;
4. save the query input as an Apify Task;
5. schedule the Task weekly or monthly;
6. compare PMID values with the previous dataset;
7. send newly observed articles to screening or alerting automation.

The Actor returns snapshots. It does not maintain a cross-run history or send alerts by itself. Use Apify schedules, webhooks, dataset APIs, or your own database for those steps.

Spreadsheet and data-pipeline integrations

You can connect the dataset to:

• Google Sheets for title and abstract screening;
• Microsoft Excel through a CSV or XLSX export;
• Make or Zapier through an Actor-run webhook;
• Python or R analysis notebooks;
• a warehouse keyed by pmid;
• a vector database for permitted abstract search;
• a review-management system that accepts tabular imports.

Use pmid as the stable deduplication key. DOI is useful but is not present on every PubMed record.

Run PubMed Articles Scraper with the API

Replace YOUR_TOKEN with an Apify API token.

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~pubmed-article-search-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["Alzheimer Disease[MeSH Terms]"],
    "minDate": "2025/01/01",
    "maxItems": 20,
    "sort": "pub_date"
  }'

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/pubmed-article-search-scraper').call({
  queries: ['CRISPR cancer'],
  maxItems: 100,
  sort: 'pub_date',
  includeAbstract: true,
  includeMeshTerms: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient(token="YOUR_TOKEN")
run = client.actor("automation-lab/pubmed-article-search-scraper").call(
    run_input={
        "queries": ["immunotherapy AND Meta-Analysis[Publication Type]"],
        "maxItems": 100,
        "sort": "pub_date",
    }
)

items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)

For asynchronous jobs, start a normal Actor run and poll its status or receive an Apify webhook instead of using the synchronous endpoint.

Use the Actor through MCP

Add the Apify MCP server to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"

Claude Desktop

Add this server object to Claude Desktop's MCP configuration:

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"
    }
  }
}

Cursor

Open Settings → MCP, add a remote HTTP server, and use:

https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper

VS Code

Add an HTTP MCP server in your VS Code MCP configuration with the same endpoint:

{
  "servers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/pubmed-article-search-scraper"
    }
  }
}

Example prompts:

• "Search PubMed for the 25 newest CRISPR delivery articles and return titles, abstracts, DOI, and MeSH terms."
• "Find randomized controlled trials in Lancet matching medicine and save the normalized dataset."
• "Run my Alzheimer Disease MeSH monitoring query from 2025 onward and summarize the new article topics."

Your MCP client still needs access to your Apify account and token according to its setup.

Limits and responsible NCBI usage

The Actor intentionally uses the official anonymous NCBI E-utilities route.

Important limits:

• a run accepts at most 20 queries and 10,000 unique articles;
• anonymous requests are throttled to three per second;
• a supplied NCBI API key enables a bounded rate below ten per second;
• larger result sets take longer because article metadata is fetched in batches;
• PubMed can return estimated or changing counts while its index updates;
• abstracts, MeSH terms, DOI, ORCID, and affiliations are not present on every article;
• full text is outside this Actor's scope;
• PubMed Central IDs indicate availability in PMC but do not make every use unrestricted.

The Actor retries temporary network errors, HTTP 429 responses, and NCBI server errors. It fails the run after bounded retries rather than silently returning a partial success as complete.

Legal and ethical use

PubMed bibliographic records are public, but article abstracts, full text, and linked publisher content may have separate rights and licenses.

You are responsible for:

• following NCBI policies and attribution guidance;
• using an NCBI API key only if you are authorized to use it;
• respecting publisher copyright and database rights;
• avoiding attempts to retrieve paywalled full text through this Actor;
• handling public author and affiliation data for a legitimate research purpose;
• complying with privacy, research, and institutional requirements that apply to your project.

This Actor does not bypass access controls and does not log into PubMed.

Troubleshooting

Why did the run return no articles?

Open the same query in PubMed and check field names, quotation marks, date limits, journal names, and publication types. Several restrictive filters are combined, so one misspelled journal can reduce the result to zero.

Why is an abstract or DOI missing?

PubMed does not index those fields for every record. The Actor preserves the article with an absent field rather than inventing data.

Why did NCBI return HTTP 429?

The Actor already throttles requests and retries transient 429 responses. Repeated failures can indicate unusual upstream load. Retry later, reduce concurrent runs, or provide your own authorized NCBI API key.

Why is publicationDate not a full ISO date?

Some records only contain a year, month, season, or date range. The Actor normalizes precise components when present and preserves PubMed's textual date when that is the only source value.

Can I request more than 10,000 records?

Not in one run. Split the research question into defensible date ranges or topic queries, then deduplicate by PMID in your destination system.

Frequently asked questions

Does this scrape PubMed Central full text?

No. It searches PubMed article records and returns metadata and abstracts indexed by PubMed. A pmcId is included when PubMed links the record to PubMed Central.

Does it support PubMed advanced search syntax?

Yes. Put supported PubMed Boolean expressions and field tags directly in queries.

Can I monitor newly indexed articles?

Yes. Save the input as a scheduled Apify Task, sort by publication date, use a date boundary, and compare PMIDs with your previous dataset. The Actor itself returns each run's snapshot and does not maintain alerts.

Are duplicate articles charged twice?

No. Within a run, repeated PMIDs across queries are saved and charged once.

Is an NCBI API key required?

No. It is optional. Anonymous runs use the official lower request limit.

Can I export to CSV or Excel?

Yes. Open the run's default dataset and select CSV or Excel, or request the corresponding dataset API format.

Related Actors

This Actor currently operates as a standalone PubMed literature-data product. The automation-lab portfolio does not yet have another active public Actor with the same biomedical article workflow, so this README does not recommend an unrelated product merely to create a cross-link.

For adjacent workflows, combine the dataset with Apify's schedules, webhooks, storage integrations, and your own screening or analysis tools.

Support

If a query behaves differently from PubMed, include:

• the Actor run URL;
• the exact input with secrets removed;
• the expected PubMed behavior;
• one PMID that demonstrates the discrepancy, if available.

Do not include your NCBI API key in a support message or dataset.