PubMed Scraper & New Article Monitor

Run your PubMed searches on a schedule through NCBI's official E-utilities API and get only the articles that are new since the last run. Every row has the abstract, authors, first affiliation, journal, MeSH terms, keywords, funding agencies, DOI and a free full-text (PubMed Central) link. Reviews and retractions are flagged.

Why this actor: PubMed e-mail alerts are hard to process. This PubMed scraper gives you the same saved search as clean rows, ready for a spreadsheet, a Slack channel or an AI summary, and remembers what it already sent you.

🇩🇪 Deutsche Version

Contents

Key features · Use cases · Quick start · Input · Output · Pricing · Integrations · Limits · FAQ · Legal and data source · Changelog · Related actors · Support

Key features

• Official NCBI E-utilities (ESearch, ESummary, EFetch): no HTML scraping. An API key is optional.
• Full PubMed query syntax: field tags, MeSH, Boolean operators. Anything that works in the PubMed search box works here.
• Monitor mode: only new PMIDs per Monitor name; duplicates across your queries are removed.
• Rich records: abstract (structured sections labelled), MeSH, keywords, first affiliation, grants, DOI and PMC links.
• Flags: isReview, isRetracted, isFreeFullText (true only when the PMC copy is no longer under embargo) and pmcEmbargoed.
• Publication-type filter, rolling look-back window or fixed dates, choice of date type and sort order.
• Respects NCBI limits: 3 requests per second, 10 with your key. If abstracts cannot be fetched, the articles are still delivered and the summary counts the missing abstracts.

Use cases

• Researchers and clinicians: a weekly digest of new papers on your topic.
• Pharma medical affairs and pharmacovigilance: literature monitoring for a product or adverse event.
• Systematic reviews: repeatable, logged searches with abstracts for screening.
• Competitive intelligence: new publications mentioning a drug, target or company.
• AI agents and RAG: fresh abstracts with MeSH terms for summarisation.

Quick start

1. Click Try for free / Start. The form is prefilled (semaglutide articles from the last 30 days), so the first run works without changes.
2. Replace the example with your own PubMed queries and adjust the filters.
3. Open the Output tab. Use the views (Overview, Abstracts and MeSH) or export JSON, CSV, Excel or HTML.
4. To monitor: switch on Only new since last run, give the watch a Monitor name, save it as a task and add a schedule (for example daily at 07:00). From the second run you receive only changes.

Input

Add one or more PubMed queries. Each query is run separately and the results are merged without duplicates.

{
  "queries": [
    "\"glucagon-like peptide-1 receptor agonists\"[MeSH] AND alcohol*[tiab]",
    "tirzepatide[tiab] AND (obesity[mh] OR \"weight loss\"[tiab])"
  ],
  "publicationTypes": [
    "Randomized Controlled Trial",
    "Review"
  ],
  "lastDays": 7,
  "maxResultsPerQuery": 200,
  "includeAbstract": true,
  "onlyNew": true,
  "stateKey": "glp1-weekly"
}

Tips: use Date from and Date to for a fixed window; they replace Look back (days). Add your e-mail as NCBI recommends. Add a free NCBI API key only for large runs; it is stored as a secret and never written to the output.

Output

Each row is one article. changeType tells you why it is in the dataset: BASELINE (first run), NEW (PMID not seen before) and SEEN (only when monitor mode is off).

Real rows from a run on 1 November 2026:

PMID	Title	First author	Journal	Published	Type	Free full text
42772688	Food noise: A theoretical review.	Wylie F	Appetite	2027-01-01	Journal Article, Review	no
42871568	The pharmacotherapy for pediatric obesity: insights and challenges fr…	Chen Z	Journal of diabetes and metabolic disorders	2026-10-12	Journal Article, Review	no
42871273	GLP-1 receptor agonists in elective spine surgery: a systematic revie…	Ferraz VR	North American Spine Society journal	2026-09-01	Journal Article, Review	yes

One full record (long values shortened):

{
  "changeType": "BASELINE",
  "query": "semaglutide[tiab]",
  "pmid": "42772688",
  "title": "Food noise: A theoretical review.",
  "authors": "Wylie F, Mahmut MK, Francis HM, Davidson T, Stevenson RJ",
  "authorCount": 5,
  "firstAuthor": "Wylie F",
  "lastAuthor": "Stevenson RJ",
  "firstAffiliation": "School of Psychological Sciences, Macquarie University, Sydney, Australia. Electronic address: fiona.wylie@mq.edu.au.",
  "journal": "Appetite",
  "journalAbbrev": "Appetite",
  "pubDate": "2027-01-01",
  "pubDateText": "2027 Jan 1",
  "epubDate": "2026 Sep 22",
  "volume": "228",
  "issue": null,
  "pages": "108791",
  "pubTypes": "Journal Article, Review",
  "isReview": true,
  "isRetracted": false,
  "language": "eng",
  "doi": "10.1016/j.appet.2026.108791",
  "pmcid": null,
  "isFreeFullText": false,
  "pmcEmbargoed": false,
  "hasAbstract": true,
  "abstract": "Food noise has emerged as a new term to describe a preoccupation with food that can be persistent, intrusive, and unwan…",
  "meshTerms": "Humans; Feeding Behavior; Food Addiction; Cues; Glucagon-Like Peptide-1 Receptor Agonists; Homeostasis; Glucagon-Like P…",
  "keywords": "Cravings; Food addiction; Food noise; Food preoccupation; Memory inhibition; Thought suppression",
  "grantAgencies": null,
  "url": "https://pubmed.ncbi.nlm.nih.gov/42772688/",
  "doiUrl": "https://doi.org/10.1016/j.appet.2026.108791",
  "pmcUrl": null,
  "scrapedAt": "2026-11-01T20:34:40.624Z"
}

A run summary is saved as RUN_SUMMARY in the run's key-value store (counts per change type, total matches per query, the query as PubMed translated it, missing abstracts).

Pricing

Pay per result: $0.75 per 1,000 articles plus $0.003 per run. Apify platform usage is included, so there is nothing else to pay. Check the Pricing tab for the current price.

Scenario	Volume	Cost
Try it: prefilled semaglutide search	100 articles, 1 run	$0.078
Daily alert for 3 queries, about 10 new papers a day	300 articles, 30 runs	$0.32
Export 5,000 articles with abstracts	5,000 articles, 1 run	$3.75

Monitor mode keeps a schedule cheap: after the first run you pay only for new articles. Apify's free plan includes $5 of monthly credit, which covers small monitors.

Integrations

• Schedules: Console → Schedules → add this actor's saved task (daily or hourly). Keep the same Monitor name so each run compares with the previous one.
• Webhooks: on the Integrations tab add a webhook for Run succeeded to POST the run to your own endpoint, then fetch rows from https://api.apify.com/v2/datasets/{datasetId}/items?format=json.
• Slack and email: add the Slack or Email integration on the Integrations tab to get a message after each run.
• Zapier, Make and n8n: use the official Apify app ("Watch actor runs" trigger, then "Get dataset items"). Filter on changeType to forward only NEW articles.
• Google Sheets: connect through Make or Zapier, or export the dataset as CSV and import it.
• MCP and AI agents: add https://mcp.apify.com?tools=heaped_qualm/pubmed-monitor to Claude, Cursor or any MCP client. The flat rows are easy for agents to read.

Python (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("heaped_qualm/pubmed-monitor").call(run_input={"queries": ["\"glucagon-like peptide-1 receptor agonists\"[MeSH] AND alcohol*[tiab]", "tirzepatide[tiab] AND (obesity[mh] OR \"weight loss\"[tiab])"], "publicationTypes": ["Randomized Controlled Trial", "Review"], "lastDays": 7, "maxResultsPerQuery": 200, "includeAbstract": true, "onlyNew": true, "stateKey": "glp1-weekly"})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["changeType"], item["pmid"], item["title"])

JavaScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('heaped_qualm/pubmed-monitor').call({"queries": ["\"glucagon-like peptide-1 receptor agonists\"[MeSH] AND alcohol*[tiab]", "tirzepatide[tiab] AND (obesity[mh] OR \"weight loss\"[tiab])"], "publicationTypes": ["Randomized Controlled Trial", "Review"], "lastDays": 7, "maxResultsPerQuery": 200, "includeAbstract": true, "onlyNew": true, "stateKey": "glp1-weekly"});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.table(items);

HTTP (waits for the run and returns the rows):

curl -X POST "https://api.apify.com/v2/acts/heaped_qualm~pubmed-monitor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["\"glucagon-like peptide-1 receptor agonists\"[MeSH] AND alcohol*[tiab]", "tirzepatide[tiab] AND (obesity[mh] OR \"weight loss\"[tiab])"], "publicationTypes": ["Randomized Controlled Trial", "Review"], "lastDays": 7, "maxResultsPerQuery": 200, "includeAbstract": true, "onlyNew": true, "stateKey": "glp1-weekly"}'

Limits

• Up to 9,999 articles per query per run, which is the ESearch limit. Split larger searches by date.
• Abstracts are as published in PubMed; some publishers do not deposit abstracts (hasAbstract is false).
• Without an API key NCBI allows 3 requests per second, so 1,000 articles with abstracts take about 10 seconds.

FAQ

Is it legal to scrape PubMed? This actor uses NCBI's official E-utilities API within its published limits and identifies itself as NCBI asks. PubMed citation data may be reused; abstracts can be copyrighted by their publishers, so check reuse rights before republishing them.

Do I need an NCBI API key? No. A free key only raises the rate limit from 3 to 10 requests per second.

Is the query syntax the same as on PubMed? Yes. The run log and RUN_SUMMARY show how PubMed translated your query, so you can check it.

What does isFreeFullText mean? The article has a PubMed Central copy that is available now. If the copy is still under embargo, pmcEmbargoed is true instead.

How do I get a full export instead of only changes? Switch off Only new since last run, or use a new Monitor name.

Why are all rows BASELINE on the first run? The first run has nothing to compare with, so everything it finds is the baseline. Later runs with the same Monitor name report only changes.

The run failed or returned nothing. What now? Read the status message and the RUN_SUMMARY record. If a query returns nothing, paste it into pubmed.ncbi.nlm.nih.gov to check the syntax. If it still looks wrong, open an issue on the Issues tab with the run link.

Legal and data source

Data source: PubMed via the NCBI E-utilities API (U.S. National Library of Medicine). Requests follow NCBI's usage guidelines (rate limits, tool name and contact e-mail). Each row links to the PubMed record. Abstracts may be subject to publisher copyright. Not medical advice.

This actor is independent and not affiliated with or endorsed by NCBI, the National Library of Medicine or the NIH. Data is provided as is, without warranty.

Changelog

• 0.3 (Nov 2026): automated test suite (unit, integration, input and schema tests, 100% line coverage); fixes: Date to alone is now respected, embargoed PMC copies are no longer flagged as free full text (new pmcEmbargoed field), an abstract download failure no longer drops the query, an invalid API key fails fast with a clear message; query translation logged; documented output schema with two views; monitor mode starts switched off in the form so test runs always show results.
• 0.2: abstracts, MeSH, keywords, affiliation and grants; publication-type filter; date range and type; optional API key; flags; de-duplication across queries.
• 0.1: first release.

Related actors

More free monitors by the same author, built on official APIs:

• ClinicalTrials.gov Scraper & Trial Monitor: ClinicalTrials.gov scraper and trial monitor
• SEC EDGAR Scraper: 8-K, 10-K, Form 4 Filings: SEC EDGAR scraper for 8-K, 10-K, 10-Q and Form 4 filings
• Greenhouse, Lever & Ashby Jobs Scraper: Greenhouse, Lever & Ashby jobs scraper and monitor
• Hacker News Scraper & Keyword Monitor: Hacker News scraper and keyword monitor

Support

Open an issue on the Issues tab of this page. Please include the run link and what you expected to see. Feature requests are welcome.

More monitors from the same author: ClinicalTrials.gov Scraper & Trial Monitor · SEC EDGAR Filings Monitor · Greenhouse, Lever, Ashby & Recruitee Jobs Monitor

---

🇩🇪 Deutsch: PubMed Scraper & Monitor für neue Artikel

Führt Ihre PubMed-Suchen nach Zeitplan über die offizielle NCBI-E-utilities-API aus und liefert nur neue Artikel seit dem letzten Lauf – mit Abstract, Autoren, Affiliation, MeSH, Keywords, Förderern, DOI und Volltext-Link.

Funktionen

• Offizielle NCBI-API, API-Schlüssel optional
• Volle PubMed-Suchsyntax
• Monitor-Modus ohne Duplikate über mehrere Suchen
• Kennzeichnung von Reviews, Rückzügen und freiem Volltext
• Einhaltung der NCBI-Limits

Anwendungsfälle

• Forschung und Klinik: wöchentlicher Literatur-Digest
• Medical Affairs und Pharmakovigilanz: Literaturüberwachung
• Systematische Reviews
• KI-Zusammenfassungen neuer Studien

Schnellstart

1. Start klicken. Das Formular ist vorausgefüllt, der erste Lauf funktioniert ohne Änderungen.
2. Eigene PubMed-Suchanfragen eintragen und Filter anpassen.
3. Ergebnisse im Tab Output ansehen oder als JSON, CSV oder Excel exportieren.
4. Für die Überwachung Only new since last run einschalten, einen Monitor-Namen vergeben, als Task speichern und einen Zeitplan anlegen. Ab dem zweiten Lauf kommen nur noch Änderungen.

Eingabebeispiel: siehe JSON oben (zwei GLP-1-Suchen, RCTs und Reviews, letzte 7 Tage).

Ausgabe: eine Zeile pro Artikel. changeType zeigt den Grund: BASELINE (erster Lauf), NEW, SEEN (nur ohne Monitor-Modus).

Preis: $0.75 pro 1.000 Artikel plus $0.003 pro Lauf, Plattformkosten inklusive. Beispiel: täglicher Alarm für 3 Suchen mit etwa 10 neuen Artikeln pro Tag ≈ $0.32 im Monat. Den aktuellen Preis zeigt der Tab Pricing.

Integrationen: Zeitpläne, Webhooks, Slack, E-Mail, Zapier, Make, n8n, Google Sheets, API (Python, JavaScript, HTTP) und MCP für KI-Agenten (https://mcp.apify.com?tools=heaped_qualm/pubmed-monitor).

Grenzen: bis 9.999 Artikel pro Suche und Lauf.

Rechtliches und Datenquelle: Quelle: PubMed über die NCBI-E-utilities (U.S. National Library of Medicine), Nutzung gemäß NCBI-Richtlinien. Abstracts können urheberrechtlich geschützt sein. Keine medizinische Beratung. Kein Bezug zu und keine Unterstützung durch NCBI, the National Library of Medicine or the NIH. Daten ohne Gewähr.

Support: Tab Issues auf dieser Seite, bitte mit Link zum Lauf.