# PubMed Scraper (`anyxsolutions/pubmed-scraper`) Actor Scrape PubMed articles by search, filtered search link or PMID: titles, full abstracts, authors with affiliations and ORCID iDs, journal details, DOIs, MeSH terms, keywords, publication types, grants and free full-text links. Clean JSON, no login, no API key, no proxy. - **URL**: https://apify.com/anyxsolutions/pubmed-scraper.md - **Developed by:** [Anyx Solutions](https://apify.com/anyxsolutions) (community) - **Categories:** AI, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ![banner](https://anyx.solutions/banners/pubmed-scraper.png) ## PubMed Scraper **Biomedical articles from PubMed — by search, filtered search link or PMID — with authors, MeSH terms, free full-text links and optional abstracts, as clean JSON.** PubMed indexes more than 38 million citations from MEDLINE, life-science journals and online books: the reference index of medicine, biology, pharmacology, public health and nursing. This Actor turns PubMed into a dataset: paste a PubMed search (with its filters) or an article link, type your own searches, or list PMIDs, and get every matching article with its title, authors with affiliations and ORCID iDs, journal, volume, issue and pages, publication date, DOI, PMCID, publication types, MeSH terms with their major-topic flag, keywords, language, grants, and a link to the free full text in PubMed Central when there is one. Abstracts (structured sections kept apart) are opt-in with `includeAbstracts`, because publishers may hold their copyright. Filter by publication date and sort by best match, recency, publication date, author or journal. There is no login and no proxy to configure, and an API key is optional. Built for systematic and literature reviews, medical-affairs and pharmacovigilance monitoring, competitive intelligence on therapies and targets, author mapping, grant analysis, and feeding article metadata into search, RAG and summarisation pipelines. ### ⚡ Quick start ```json { "startUrls": [ { "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review" } ], "maxItems": 5 } ``` Or run your own searches with PubMed field tags and a date range: ```json { "searchTerms": ["sickle cell disease[mh] AND gene therapy[tiab]"], "dateFrom": "2024-01-01", "sortBy": "pub_date", "maxItems": 100 } ``` ### 🧩 Input | Field | Type | Required | Default | Description | | -------------------- | ------- | -------- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `startUrls` | array | no | — | PubMed URLs: searches (`pubmed.ncbi.nlm.nih.gov/?term=…`, with their filters and sort) or articles (`pubmed.ncbi.nlm.nih.gov//`). | | `maxItems` | integer | no | — | Maximum number of articles per start URL or search term. The input form starts at 5; leave it empty to get every match (PubMed returns at most 9,999 articles per search). | | `searchTerms` | array | no | — | Searches, one per line, written as on the PubMed site. Field tags work, e.g. `crispr[ti] AND doudna[au]` or `asthma[mh] AND review[pt]`. | | `pmids` | array | no | — | PubMed IDs to look up, such as `33301246`. Article links work too. Every listed article is returned, whatever `maxItems` says. | | `sortBy` | string | no | `"relevance"` | Sort search-term results by `relevance` (best match), `most_recent` (recently added), `pub_date`, `first_author` or `journal`. | | `dateFrom` | string | no | — | Only search-term results published on or after this date (`YYYY-MM-DD`). | | `dateTo` | string | no | — | Only search-term results published on or before this date (`YYYY-MM-DD`). | | `includeAbstracts` | boolean | no | `false` | Add each article's abstract. Abstracts may be copyrighted by their publishers; turn on only if your use is permitted. | | `apiKey` | string | no | — | Optional free NCBI API key. Raises the request rate from three to ten a second, so large runs finish sooner. | | `proxyConfiguration` | object | no | `{ "useApifyProxy": false }` | Proxy settings. PubMed needs no proxy. | Add at least one start URL, search term or PMID. `sortBy`, `dateFrom` and `dateTo` apply to the search terms; a search URL keeps the sort and filters it was saved with. **Search URL filters.** Set up a search on pubmed.ncbi.nlm.nih.gov, tick the filters you want, and paste the address. The Actor keeps the publication-year or custom date range, article types (review, clinical trial, meta-analysis, systematic review and the rest), text availability (abstract, free full text, full text), associated data, species, sex, age, language, MEDLINE and exclude-preprints filters, and the sort (best match, most recent, publication date, first author, journal). Any filter it cannot read is named in the log and left out. ### 📤 Output Each article is one dataset row. #### Fields | Field | Type | Description | | ------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pmid` | string | PubMed ID. | | `url` | string | The article's PubMed page. | | `doi` | string | DOI, when the article has one. | | `pmcid` | string | PubMed Central ID, such as `PMC9294086`, when the full text is in PubMed Central. | | `title` | string | Article title (book documents use the chapter or book title). | | `abstractText` | string | The full abstract, only with `includeAbstracts` on (`null` otherwise). Structured abstracts keep their section labels (`BACKGROUND: …`), one paragraph per section. | | `abstractSections` | array | The abstract's sections, each with `label`, NLM's normalised `category` and `text`; `null` unless `includeAbstracts` is on. | | `authors` | array | Authors in order, each with `name`, `lastName`, `foreName`, `initials`, `affiliations` and `orcid`. Group authors such as trial consortia carry only `name`. | | `journal` | object | `title`, `isoAbbreviation`, `issn`, `volume`, `issue` and `pages`. | | `publicationDate` | string | Publication date as precise as PubMed gives it: `2020-12-31`, `2022-04` or `2016`. | | `publicationTypes` | array | Publication types, such as `Review`, `Randomized Controlled Trial` or `Meta-Analysis`. | | `meshTerms` | array | MeSH headings, each with `descriptor`, `descriptorId`, `major` (a main topic of the article) and `qualifiers`. | | `keywords` | array | Author and publisher keywords. | | `language` | string | Language of the article as a three-letter code, such as `eng`. | | `grants` | array | Funding, each with `grantId`, `acronym`, `agency` and `country`. | | `freeFullTextUrl` | string | Link to the full text in PubMed Central, when the article has a PMCID. | | `searchQuery` | string | The PubMed query that found the article, filters included; `null` for articles added by PMID or link. |
Example output ```json { "pmid": "35165856", "url": "https://pubmed.ncbi.nlm.nih.gov/35165856/", "doi": "10.1007/s13311-022-01197-9", "pmcid": "PMC9294086", "title": "CRISPR-Cas9 Gene Therapy for Duchenne Muscular Dystrophy.", "abstractText": null, "abstractSections": null, "authors": [ { "name": "Cedric Happi Mbakam", "lastName": "Happi Mbakam", "foreName": "Cedric", "initials": "C", "affiliations": [ "CHU de Québec Research Center - Laval University, Québec, Canada.", "Department of Molecular Medicine, Faculty of Medicine, Laval University, Québec City, Québec, Canada." ], "orcid": null }, { "name": "Jacques P Tremblay", "lastName": "Tremblay", "foreName": "Jacques P", "initials": "JP", "affiliations": [ "CHU de Québec Research Center - Laval University, Québec, Canada. Jacques-P.Tremblay@crchul.ulaval.ca.", "Department of Molecular Medicine, Faculty of Medicine, Laval University, Québec City, Québec, Canada. Jacques-P.Tremblay@crchul.ulaval.ca." ], "orcid": "0000-0001-9404-9195" } ], "journal": { "title": "Neurotherapeutics : the journal of the American Society for Experimental NeuroTherapeutics", "isoAbbreviation": "Neurotherapeutics", "issn": "1878-7479", "volume": "19", "issue": "3", "pages": "931-941" }, "publicationDate": "2022-04", "publicationTypes": [ "Journal Article", "Review", "Research Support, Non-U.S. Gov't" ], "meshTerms": [ { "descriptor": "CRISPR-Cas Systems", "descriptorId": "D064113", "major": false, "qualifiers": ["genetics"] }, { "descriptor": "Muscular Dystrophy, Duchenne", "descriptorId": "D020388", "major": true, "qualifiers": ["genetics", "metabolism", "therapy"] } ], "keywords": [ "CRISPR-Cas", "DMD gene", "Duchenne muscular dystrophy", "Dystrophin", "Gene therapy" ], "language": "eng", "grants": [ { "grantId": null, "acronym": null, "agency": "CIHR", "country": "Canada" } ], "freeFullTextUrl": "https://pmc.ncbi.nlm.nih.gov/articles/PMC9294086/", "searchQuery": "(crispr gene therapy) AND review[Filter]" } ```
### 💡 Use cases - **Systematic and literature reviews** — pull every article matching a search strategy, with MeSH terms and publication types, into a spreadsheet or screening tool, adding abstracts where your use is permitted. - **Medical affairs and pharmacovigilance** — a weekly feed of new publications on your products, competitors, targets or adverse events. - **Competitive intelligence** — track the trials, reviews and case reports published on a therapy area or mechanism. - **Author and expert mapping** — find the active authors in a field, with their institutions and ORCID iDs. - **Grant and funding analysis** — see which agencies fund the work on a topic. - **RAG and summarisation** — feed article metadata and free full-text links into search, embedding or LLM pipelines. ### ❓ FAQ - **Do I need an NCBI account or API key?** No. The Actor works without one. A free NCBI API key, entered in `apiKey`, makes large runs about three times faster. - **Do I need a proxy?** No. It runs without one by default. - **Why is it not faster?** NCBI allows three requests a second without an API key, and the Actor keeps to that. Each request returns up to 200 full articles, so even large searches finish in minutes. - **Why does a search stop at 9,999 articles?** PubMed returns only the first 9,999 matches of one search. Split a larger search by year with `dateFrom` and `dateTo`, or use several search URLs with year filters. - **Does it download the full text?** No. Every article in PubMed Central carries `freeFullTextUrl`, so you can open or fetch the ones you need. - **Why are the MeSH terms empty?** New articles wait weeks or months for MEDLINE indexing, and some journals and book documents are never indexed with MeSH. Keywords are often present instead. - **Why is the abstract empty?** Abstracts are off by default: `abstractText` and `abstractSections` are `null` unless you turn on `includeAbstracts`. Abstracts may be copyrighted by their publishers, so turn it on only if your use is permitted. With it on, letters, editorials, comments and many older articles still have no abstract in PubMed. - **Can I search one author or journal?** Yes. Use PubMed field tags in `searchTerms`, for example `doudna ja[au]` or `"n engl j med"[ta]`. ### 🔗 More scrapers by Anyx - [arXiv Scraper](https://apify.com/anyxsolutions/arxiv-scraper) - [Crossref Scraper](https://apify.com/anyxsolutions/crossref-scraper) - [OpenAlex Scraper](https://apify.com/anyxsolutions/openalex-scraper) - [Google Search Scraper](https://apify.com/anyxsolutions/google-search-scraper) ### 🤝 Anyx Solutions We build custom scrapers and data-extraction pipelines. - Email: tantosthor@gmail.com ### 🖼 Image credit Image credit: [pubmed.ncbi.nlm.nih.gov](https://pubmed.ncbi.nlm.nih.gov/) # Actor input Schema ## `startUrls` (type: `array`): PubMed URLs, for example a search such as https://pubmed.ncbi.nlm.nih.gov/?term=crispr\&filter=pubt.review (its year, article type, text availability, language, species, sex and age filters and its sort are kept) or an article such as https://pubmed.ncbi.nlm.nih.gov/33301246/. ## `maxItems` (type: `integer`): Maximum number of articles to return per start URL or search term. Leave empty to get every match (PubMed returns at most 9,999 articles per search). ## `searchTerms` (type: `array`): Searches to run on PubMed, one per line, exactly as you would type them on the site, for example gene therapy sickle cell. Field tags work too, e.g. crispr\[ti] AND doudna\[au] or asthma\[mh] AND review\[pt]. ## `pmids` (type: `array`): PubMed IDs to look up, one per line, for example 33301246. Article links work too. Every listed article is returned, whatever the maximum. ## `sortBy` (type: `string`): Order of the search-term results: best match, most recently added, publication date (newest first), first author or journal name. ## `dateFrom` (type: `string`): Only return search-term results published on or after this date, for example 2024-01-01. ## `dateTo` (type: `string`): Only return search-term results published on or before this date, for example 2025-12-31. ## `includeAbstracts` (type: `boolean`): Add each article's abstract to the output. Abstracts may be copyrighted by their publishers; turn on only if your use is permitted. ## `apiKey` (type: `string`): Optional. Your free NCBI API key from your NCBI account settings, for example a 36-character key. It lets the Actor make ten requests a second instead of three, so large runs finish sooner. ## `proxyConfiguration` (type: `object`): Proxy settings. PubMed needs no proxy; the Actor keeps to NCBI's request rate. ## Actor input object example ```json { "startUrls": [ { "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review" } ], "maxItems": 5, "sortBy": "relevance", "includeAbstracts": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review" } ], "maxItems": 5 }; // Run the Actor and wait for it to finish const run = await client.actor("anyxsolutions/pubmed-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review" }], "maxItems": 5, } # Run the Actor and wait for it to finish run = client.actor("anyxsolutions/pubmed-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://pubmed.ncbi.nlm.nih.gov/?term=crispr+gene+therapy&filter=years.2020-2024&filter=pubt.review" } ], "maxItems": 5 }' | apify call anyxsolutions/pubmed-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,anyxsolutions/pubmed-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/Ve4Ht8dokLspF6GMq/builds/UGOIjHAxdh90jSUip/openapi.json