PubMed Scraper: Papers, Abstracts, MeSH Terms and Citation Metrics

This PubMed scraper turns the biomedical literature into clean rows. Search by topic, author, journal, institution, MeSH term or publication type and get one row per paper with the title, the full abstract, the authors with their institutions and countries, the MeSH terms, keywords, grants, clinical trial registration numbers and the links to the paper, the DOI and the free full text.

Each paper also comes with the citation metrics of the National Institutes of Health: number of citations, citations per year, relative citation ratio, percentile among papers of the same age and field, how many clinical articles cite it and a translation score that shows how close the work is to human use. That is the information that literature reviews, competitive intelligence and research funding decisions need, without joining several sources by hand.

Pharma and biotech analysts, medical affairs teams, systematic reviewers, research offices, health tech builders, journalists and students use it to find and rank evidence.

What can you do with this PubMed scraper?

• Pull all papers on a drug, a disease or a method, with abstracts
• Keep only randomized trials, meta-analyses or systematic reviews
• Rank papers by relative citation ratio or citations per year
• Find the most active institutions and countries on a topic
• Follow what one author or one journal publishes
• Look up a list of PubMed ids or DOIs with all their data
• Find papers that cite or register a clinical trial number
• Spot retracted papers and papers with a retraction notice
• Count how many papers a topic had each year and its growth
• Export to CSV or Excel, or send new papers to your own tools

What data can you extract from PubMed?

Mode	One row is	Highlights
Search articles	One paper	PubMed id, DOI, PMC id, title, abstract and its sections, journal, abbreviation, ISSN, country, volume, issue, pages, publication date, electronic date, status, authors with institutions, ORCID and country, first and last author, countries, MeSH headings with qualifiers and main topics, keywords, publication types, review, trial, randomized, meta-analysis, case report and retraction flags, trial registrations, grants and funding agencies, chemicals, number of references, links, citation count, citations per year, relative citation ratio, NIH percentile, clinical citations, translation score, human, animal and molecular scores
Look up	One paper by PubMed id or DOI	Same fields
Trends	One search term and one year	Articles, change from the year before, share of all PubMed papers, total in the year, partial year flag

How to scrape PubMed

1. Choose Search articles, Look up or Publication trends.
2. Type search terms, one per line, using plain words or PubMed syntax.
3. Add filters: authors, journals, MeSH terms, institutions, publication types, dates.
4. Set Max Items and click Start.
5. Export the dataset as CSV, Excel or from the Apify console.

Input example

{
  "mode": "articles",
  "searchTerms": ["semaglutide cardiovascular outcomes"],
  "publicationTypes": ["Randomized Controlled Trial"],
  "dateFrom": "2023",
  "includeCitationMetrics": true,
  "maxItems": 200
}

Paper output example

{
  "pmid": "33301246",
  "doi": "10.1056/NEJMoa2034577",
  "title": "Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine",
  "journal": "The New England journal of medicine",
  "publicationDate": "2020-12-31",
  "authorCount": 30,
  "referenceCount": 13,
  "citationCount": 11726,
  "relativeCitationRatio": 600.78,
  "nihPercentile": 100,
  "clinicalArticlesCiting": 610
}

Trends output example

{
  "searchTerm": "glp-1 receptor agonist",
  "year": 2023,
  "articles": 1494,
  "changeFromPreviousYearPct": 15.8,
  "shareOfPubMedPct": 0.088
}

How much does it cost to scrape PubMed?

You pay per result: $17 per 1,000 rows plus a tiny start fee. The free plan returns up to 10 rows per run. A trends table is one row per term and year, so it costs a few cents.

Tips for better results

• You can use the PubMed search syntax in a term, for example heart failure[ti] AND humans[mh]. Plain words work too.
• PubMed returns at most 10,000 papers per search. For more, the Actor cuts the search into years when you ask for more than 9,000 rows.
• Citation metrics come from the NIH database and cover papers indexed in PubMed. Very new papers can have no relative citation ratio yet.
• Countries are read from the last part of each affiliation text, so some papers have none.
• Author names are the public names printed in the paper. Emails are not collected.
• A free NCBI access code speeds up large runs from 3 to 10 requests per second. Paste it in the optional field.
• The publication date uses the date of the journal issue. The electronic date shows when the paper first appeared online.
• Review and meta-analysis flags come from the publication types that PubMed assigns, which are added some weeks after the paper appears.

Who uses PubMed data?

• Pharma, biotech and medical affairs teams map evidence and leading experts.
• Systematic reviewers collect and screen candidate papers.
• Research offices and funders measure output and impact.
• Health tech and AI builders feed abstracts into search and language models.
• Journalists and students find and compare studies.

Automate and connect

Schedule the Actor every week with the publication date of the last seven days and send new papers to Slack, Google Sheets, Zapier, Make or n8n, or trigger a webhook when a run ends.

FAQ

Do I need an account or a login? No. The Actor works without any login. A free NCBI access code is optional.

Does it return the full text of papers? No. It returns the abstract and the links. The free full text link is in the row when it exists.

What is the relative citation ratio? A measure of the influence of a paper compared with other papers of the same field and year. A value of 1 is the average.

Can I search by clinical trial number? Yes. Put the NCT number in a search term.

How current is the data? New papers appear in PubMed within days. Citation metrics are refreshed regularly by the NIH.

Is this medical advice? No. It is a list of published research. Ask a doctor about treatment.

Legal note

This Actor collects information that is publicly visible on the website. Abstracts remain the property of their publishers. Check the source site's terms and your local rules before using the data.

Support

Missing a field or found a bug? Open an issue from the Actor page.