PubMed Scraper

Biomedical articles from PubMed — by search, filtered search link or PMID — with authors, MeSH terms, free full-text links and optional abstracts, as clean JSON.

PubMed indexes more than 38 million citations from MEDLINE, life-science journals and online books: the reference index of medicine, biology, pharmacology, public health and nursing. This Actor turns PubMed into a dataset: paste a PubMed search (with its filters) or an article link, type your own searches, or list PMIDs, and get every matching article with its title, authors with affiliations and ORCID iDs, journal, volume, issue and pages, publication date, DOI, PMCID, publication types, MeSH terms with their major-topic flag, keywords, language, grants, and a link to the free full text in PubMed Central when there is one. Abstracts (structured sections kept apart) are opt-in with includeAbstracts, because publishers may hold their copyright. Filter by publication date and sort by best match, recency, publication date, author or journal. There is no login and no proxy to configure, and an API key is optional. Built for systematic and literature reviews, medical-affairs and pharmacovigilance monitoring, competitive intelligence on therapies and targets, author mapping, grant analysis, and feeding article metadata into search, RAG and summarisation pipelines.

⚡ Quick start

{
  "startUrls": [
    {
      "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review"
    }
  ],
  "maxItems": 5
}

Or run your own searches with PubMed field tags and a date range:

{
  "searchTerms": ["sickle cell disease[mh] AND gene therapy[tiab]"],
  "dateFrom": "2024-01-01",
  "sortBy": "pub_date",
  "maxItems": 100
}

🧩 Input

Field	Type	Required	Default	Description
startUrls	array	no	—	PubMed URLs: searches (pubmed.ncbi.nlm.nih.gov/?term=…, with their filters and sort) or articles (pubmed.ncbi.nlm.nih.gov/<PMID>/).
maxItems	integer	no	—	Maximum number of articles per start URL or search term. The input form starts at 5; leave it empty to get every match (PubMed returns at most 9,999 articles per search).
searchTerms	array	no	—	Searches, one per line, written as on the PubMed site. Field tags work, e.g. crispr[ti] AND doudna[au] or asthma[mh] AND review[pt].
pmids	array	no	—	PubMed IDs to look up, such as 33301246. Article links work too. Every listed article is returned, whatever maxItems says.
sortBy	string	no	"relevance"	Sort search-term results by relevance (best match), most_recent (recently added), pub_date, first_author or journal.
dateFrom	string	no	—	Only search-term results published on or after this date (YYYY-MM-DD).
dateTo	string	no	—	Only search-term results published on or before this date (YYYY-MM-DD).
includeAbstracts	boolean	no	false	Add each article's abstract. Abstracts may be copyrighted by their publishers; turn on only if your use is permitted.
apiKey	string	no	—	Optional free NCBI API key. Raises the request rate from three to ten a second, so large runs finish sooner.
proxyConfiguration	object	no	{ "useApifyProxy": false }	Proxy settings. PubMed needs no proxy.

Add at least one start URL, search term or PMID. sortBy, dateFrom and dateTo apply to the search terms; a search URL keeps the sort and filters it was saved with.

Search URL filters. Set up a search on pubmed.ncbi.nlm.nih.gov, tick the filters you want, and paste the address. The Actor keeps the publication-year or custom date range, article types (review, clinical trial, meta-analysis, systematic review and the rest), text availability (abstract, free full text, full text), associated data, species, sex, age, language, MEDLINE and exclude-preprints filters, and the sort (best match, most recent, publication date, first author, journal). Any filter it cannot read is named in the log and left out.

📤 Output

Each article is one dataset row.

Fields

Field	Type	Description
pmid	string	PubMed ID.
url	string	The article's PubMed page.
doi	string	DOI, when the article has one.
pmcid	string	PubMed Central ID, such as PMC9294086, when the full text is in PubMed Central.
title	string	Article title (book documents use the chapter or book title).
abstractText	string	The full abstract, only with includeAbstracts on (null otherwise). Structured abstracts keep their section labels (BACKGROUND: …), one paragraph per section.
abstractSections	array	The abstract's sections, each with label, NLM's normalised category and text; null unless includeAbstracts is on.
authors	array	Authors in order, each with name, lastName, foreName, initials, affiliations and orcid. Group authors such as trial consortia carry only name.
journal	object	title, isoAbbreviation, issn, volume, issue and pages.
publicationDate	string	Publication date as precise as PubMed gives it: 2020-12-31, 2022-04 or 2016.
publicationTypes	array	Publication types, such as Review, Randomized Controlled Trial or Meta-Analysis.
meshTerms	array	MeSH headings, each with descriptor, descriptorId, major (a main topic of the article) and qualifiers.
keywords	array	Author and publisher keywords.
language	string	Language of the article as a three-letter code, such as eng.
grants	array	Funding, each with grantId, acronym, agency and country.
freeFullTextUrl	string	Link to the full text in PubMed Central, when the article has a PMCID.
searchQuery	string	The PubMed query that found the article, filters included; null for articles added by PMID or link.

💡 Use cases

• Systematic and literature reviews — pull every article matching a search strategy, with MeSH terms and publication types, into a spreadsheet or screening tool, adding abstracts where your use is permitted.
• Medical affairs and pharmacovigilance — a weekly feed of new publications on your products, competitors, targets or adverse events.
• Competitive intelligence — track the trials, reviews and case reports published on a therapy area or mechanism.
• Author and expert mapping — find the active authors in a field, with their institutions and ORCID iDs.
• Grant and funding analysis — see which agencies fund the work on a topic.
• RAG and summarisation — feed article metadata and free full-text links into search, embedding or LLM pipelines.

❓ FAQ

• Do I need an NCBI account or API key? No. The Actor works without one. A free NCBI API key, entered in apiKey, makes large runs about three times faster.
• Do I need a proxy? No. It runs without one by default.
• Why is it not faster? NCBI allows three requests a second without an API key, and the Actor keeps to that. Each request returns up to 200 full articles, so even large searches finish in minutes.
• Why does a search stop at 9,999 articles? PubMed returns only the first 9,999 matches of one search. Split a larger search by year with dateFrom and dateTo, or use several search URLs with year filters.
• Does it download the full text? No. Every article in PubMed Central carries freeFullTextUrl, so you can open or fetch the ones you need.
• Why are the MeSH terms empty? New articles wait weeks or months for MEDLINE indexing, and some journals and book documents are never indexed with MeSH. Keywords are often present instead.
• Why is the abstract empty? Abstracts are off by default: abstractText and abstractSections are null unless you turn on includeAbstracts. Abstracts may be copyrighted by their publishers, so turn it on only if your use is permitted. With it on, letters, editorials, comments and many older articles still have no abstract in PubMed.
• Can I search one author or journal? Yes. Use PubMed field tags in searchTerms, for example doudna ja[au] or "n engl j med"[ta].

🔗 More scrapers by Anyx

• arXiv Scraper
• Crossref Scraper
• OpenAlex Scraper
• Google Search Scraper

🤝 Anyx Solutions

We build custom scrapers and data-extraction pipelines.

• Email: tantosthor@gmail.com

🖼 Image credit

Image credit: pubmed.ncbi.nlm.nih.gov