bioRxiv Scraper · Preprints, Authors, Categories & Versions

Extract preprints, authors, categories, versions, licensing, and abstracts from bioRxiv and medRxiv repositories using the bioRxiv scraper. Query scientific literature across biological, medical, and health sciences with flexible date windows, category filters, and keyword search via direct HTTP API access.

Maintained by reapX. Every row cites the Apify run that produced it — nothing is inferred, modelled or filled in, and a field absent from the source is absent from the row. The extracted archive for this source is browsable at reapx.dev/data/biorxiv-scraper/ and mirrored as an open dataset on Hugging Face and Kaggle. Questions: reapxdev@proton.me

---

Features & Capabilities

• Dual Repository Support: Seamlessly fetch manuscripts from both bioRxiv (biological sciences) and medRxiv (medical, clinical, and health sciences).
• Comprehensive Metadata Extraction: Capture paper DOIs, normalized paper IDs, full titles, authors, corresponding authors, institutional affiliations, posting dates, manuscript versions, subject categories, article types, open-access licenses, funding statements, and full abstracts.
• Granular Date Window Filtering: Target specific publication date ranges using standard YYYY-MM-DD start and end dates.
• Subject Category Filtering: Isolate research in neuroscience, bioinformatics, genetics, cell biology, immunology, epidemiology, infectious diseases, clinical oncology, health informatics, and over 60 specific subject categories.
• Keyword & Author Search: Filter preprints by specific keywords in titles/abstracts (e.g. CRISPR, deep learning, mRNA vaccine, GWAS) or author names.
• Schema-Compliant Entity Routing: Every row includes canonical doi and slugified paperId fields matching standard entity page resolution requirements.
• Pay-Per-Event (PPE) Pricing: Pay only per successful preprint record written to the default dataset. Rate-limited requests, 429 backoffs, and empty searches are never billed.
• High Throughput & Reliability: Built using light HTTP client requests with automatic exponential backoff on 429 and 5xx responses.

---

⬇️ Input

Configure the bioRxiv scraper using the parameters below. All parameters carry working defaults or prefilled values for immediate execution.

Input Field	Type	Required	Default / Prefill	Description
server	String	No	biorxiv	Target repository server: biorxiv for biological sciences or medrxiv for medical sciences.
fromDate	String	No	2024-01-01	Start date for posting date window (YYYY-MM-DD).
toDate	String	No	2024-01-05	End date for posting date window (YYYY-MM-DD).
category	String	No	Empty	Subject category filter (e.g. neuroscience, bioinformatics, epidemiology).
term	String	No	Empty	Keyword phrase search filter matching paper title and abstract text.
author	String	No	Empty	Author name filter string matching primary or co-authors.
maxPapers	Integer	Yes	50	Maximum number of preprint records to extract (1 to 10,000).

Example Input Configurations

1. Basic bioRxiv Neuroscience Preprints

{
  "server": "biorxiv",
  "fromDate": "2024-01-01",
  "toDate": "2024-01-05",
  "category": "neuroscience",
  "maxPapers": 50
}

2. medRxiv Epidemiology & Infectious Diseases Search

{
  "server": "medrxiv",
  "fromDate": "2024-01-01",
  "toDate": "2024-01-10",
  "category": "epidemiology",
  "term": "outbreak",
  "maxPapers": 100
}

3. CRISPR & Gene Editing Research Across Repositories

{
  "server": "biorxiv",
  "fromDate": "2024-01-01",
  "toDate": "2024-01-15",
  "term": "CRISPR",
  "maxPapers": 100
}

---

⬆️ Output

All extracted records are pushed directly to the run's default dataset in real time.

Dataset Field Schema

Field Name	Type	Description	Example
doi	String	Digital Object Identifier assigned to the preprint.	10.1101/2023.12.30.573731
paperId	String	Normalized identifier with colons replaced by hyphens for entity page addressing.	10.1101-2023.12.30.573731
title	String	Full scientific manuscript title.	KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents
authors	String	Semicolon-separated list of listed authors.	Sinha, A. K.; Lee, C.; Holt, J. C.
authorCorresponding	String	Full name of corresponding author.	Joseph C Holt
authorCorrespondingInstitution	String	Institutional affiliation of corresponding author.	Dept. of Pharmacology, Univ. of Rochester
date	String	Manuscript posting date (YYYY-MM-DD).	2024-01-01
version	String	Preprint manuscript version iteration.	1
type	String	Article submission classification type.	new results
license	String	Open-access licensing code.	cc_by_nc_nd
category	String	Subject area research category.	neuroscience
jatsxml	String	Direct URL link to full JATS XML source file.	https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml
abstract	String	Complete abstract text.	Primary vestibular afferents transmit information...
funder	String	Funding agency or grant information.	NIH R01 DC00429
published	String	Journal DOI reference if peer-reviewed and published.	10.1016/j.neuron.2024.03.012
server	String	Repository server name (bioRxiv or medRxiv).	bioRxiv
url	String	Direct web URL to preprint webpage.	https://www.biorxiv.org/content/10.1101/2023.12.30.573731v1

Sample JSON Output Record

{
  "doi": "10.1101/2023.12.30.573731",
  "paperId": "10.1101-2023.12.30.573731",
  "title": "KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents in mammals",
  "authors": "Sinha, A. K.; Lee, C.; Holt, J. C.",
  "authorCorresponding": "Joseph C Holt",
  "authorCorrespondingInstitution": "Dept. of Pharmacology and Physiology, Univ. of Rochester, Rochester, New York 14642",
  "date": "2024-01-01",
  "version": "1",
  "type": "new results",
  "license": "cc_no",
  "category": "neuroscience",
  "jatsxml": "https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml",
  "abstract": "Primary vestibular afferents transmit information from hair cells about head position and movement to the CNS, which is critical for maintaining balance, gaze stability and spatial navigation...",
  "funder": "NA",
  "published": "NA",
  "server": "bioRxiv",
  "url": "https://www.biorxiv.org/content/10.1101/2023.12.30.573731"
}

Dataset Named Views

The dataset schema includes three pre-configured views tailored for different analytical workflows:

1. Overview (overview): Columns doi, paperId, title, server, category, date, version, url. Ideal for rapid screening.
2. Authors & Affiliations (authors_and_affiliations): Columns paperId, title, authors, authorCorresponding, authorCorrespondingInstitution, funder. Focused on bibliometric and institutional research.
3. Metadata & Licensing (metadata_and_licensing): Columns paperId, title, doi, server, category, type, license, published, jatsxml. Built for open access tracking and XML corpus pipelines.

---

How it works

The bioRxiv scraper connects directly to the official bioRxiv details REST API (https://api.biorxiv.org/details/{server}/{from}/{to}/{cursor}).

1. API Handshake: Validates input parameters, target server (biorxiv or medrxiv), and date window.
2. Paginated Retrieval: Issues lightweight HTTP GET requests using cursor pagination (30 records per request window).
3. Client-side Filtering: Filters records in memory by subject category, author name, or title/abstract keywords if specified.
4. Resilience & Backoff: Automatically handles rate limits (HTTP 429) and server glitches (HTTP 5xx) with exponential backoff (2s, 4s, 8s, 16s) up to 5 retries.
5. Pay-Per-Event Charging: Charges exactly $0.002 per valid preprint record via Actor.charge() immediately before dataset push.
6. Streaming Dataset Output: Emits items continuously to the run's default dataset so no data is lost in the event of run interruption.

---

Pay-Per-Event Pricing & Volume Discounts

This actor uses Pay-Per-Event (PPE) pricing. You are charged only for actual preprint records returned and stored in your dataset.

• Base Price: $0.002 per preprint record.
• Platform Usage: Included. Compute usage is absorbed into the event price.
• Tiered Volume Discounts: Automatically applied based on your monthly Apify plan tier:
  • FREE: $0.0020 / record
  • BRONZE: $0.0018 / record (10% discount)
  • SILVER: $0.0015 / record (25% discount)
  • GOLD: $0.0013 / record (35% discount)
  • PLATINUM: $0.0010 / record (50% discount)
  • DIAMOND: $0.0008 / record (60% discount)

---

Use Cases

• Biomedical Literature Mining: Track cutting-edge preprints across cancer research, genomics, neurobiology, and immunology before formal journal peer review.
• Epidemiology & Outbreak Surveillance: Monitor real-time public health studies, epidemiological models, and infectious disease findings on medRxiv.
• AI & LLM Training Datasets: Aggregate high-quality scientific abstracts and JATS XML links for fine-tuning domain-specific language models.
• Competitive Intelligence: Follow research output from academic labs and biotechnology companies by tracking author affiliations and funding grants.

---

❓ FAQ

Q: Do I need an API key or account to use bioRxiv Scraper?

No. The scraper accesses the open bioRxiv / medRxiv public API directly over HTTP. No external account or API key is required.

Q: Which preprint servers are supported?

Both biorxiv (bioRxiv for biological sciences) and medrxiv (medRxiv for medical and health sciences) are fully supported.

Q: How are date bounds formatted?

Use standard YYYY-MM-DD dates (e.g. 2024-01-01). The bioRxiv API filters manuscripts posted within the date window.

Q: What happens if a search returns zero results?

If no preprints match your search filters or date range, the actor completes cleanly with 0 rows emitted and charges 0 USD.

Q: Can I export results to CSV, Excel, or JSON?

Yes. All results saved to the default dataset can be downloaded from the Apify Console or API in JSON, CSV, Excel, XML, or HTML table format.

---

💬 Your feedback

Have suggestions, feature requests, or bug reports? We continuously improve our scrapers based on user feedback. Reach out directly via email at reapxdev@proton.me or visit reapx.dev.

---

Disclaimer: Unofficial - not affiliated with bioRxiv or medRxiv. Collects public data only. reapx. Contact reapxdev@proton.me.

🧪 Example input

A real, runnable configuration — this is an actual input this Actor has run with.

{
  "server": "biorxiv",
  "fromDate": "2024-01-01",
  "toDate": "2024-01-05",
  "maxPapers": 50,
  "category": "neuroscience"
}

📄 Sample output

One real row from a real run of this Actor, unedited.

{
  "doi": "10.1101/2023.12.30.573731",
  "paperId": "10.1101/2023.12.30.573731",
  "title": "KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents in mammals",
  "authors": "Sinha, A. K.; Lee, C.; Holt, J. C.",
  "authorCorresponding": "Joseph C Holt",
  "authorCorrespondingInstitution": "Dept. of Pharmacology and Physiology, Univ. of Rochester, Rochester, New York 14642",
  "date": "2024-01-01",
  "version": "1",
  "type": "new results",
  "license": "cc_no",
  "category": "neuroscience",
  "jatsxml": "https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml",
  "abstract": "Primary vestibular afferents transmit information from hair cells about head position and movement to the CNS, which is critical for maintaining balance, gaze stability and spatial navigation. The CNS, in turn, modulates hair cells and afferents via the efferent vestibular system (EVS) and its activation of several cholinergic signaling mechanisms. Electrical stimulation of EVS neurons gives rise to three kinetically- and mechanistically-distinct afferent responses including a slow excitation, a fast excitation, and a fast inhibition. EVS-mediated slow excitation is attributed to odd-numbered muscarinic acetylcholine receptors (mAChRs) on the afferent whose activation leads to the closure of a potassium conductance and increased afferent discharge.\n\nLikely effector candidates include low-threshold, voltage-gated potassium channels belonging to the KCNQ (Kv7.X) family, which are involved in neuronal excitability across the nervous system and are subject to mAChR modulation. Specifically, KCNQ2/3 heteromeric channels may be the molecular correlates for the M-current, a potassium current that is blocked following the activation of odd-numbered mAChRs. To this end, multiple members of the KCNQ channel family, including KCNQ2 and KCNQ3, are localized to several microdomains within vestibular afferent endings, where they influence afferent excitability and could be targeted by EVS neurons. Additionally, the relative expression of KCNQ subunits appears to vary across the sensory epithelia and among different afferent types. However, it is unclear which KCNQ channel subunits are targeted by mAChR activation and whether that also varies among different afferent classes. Here we show that EVS-mediated slow excitation is blocked and enhanced by the non-selective KCNQ channel blocker XE991 and opener retigabine, respectively. Using KCNQ subunit-selective drugs, we observed that a KCNQ2 blocker blocks the slow response in irregular afferents, while a KCNQ2/3 opener enhances slow responses in regular afferents. The KCNQ2 blockers did not appear to affect resting afferent discharge rates, while KCNQ2/3 or KCNQ2/4 openers decreased afferent excitability. Here, we show pharmacological evidence that KCNQ2/3 subunits are likely targeted by mAChR activation in mammalian vestibular afferents. Additionally, we show that KCNQ3 KO mice have altered resting discharge rate as well as EVS-mediated slow response. These data together suggest that KCNQ channels play a role in slow response and discharge rate of vestibular afferents, which can be modulated

⚠️ Run outcomes and error handling

This Actor reports what happened in the run's status message, and it always keeps whatever it collected. These are the outcomes you can get and what each one means.

Outcome	What it means
Success	Rows were returned and you were charged record-returned at $0.002 per row.
No matches	The source returned nothing for your filters. Nothing is charged. Widen the date window or drop a filter.

What is guaranteed either way

• Every row is pushed as it is built, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not.
• A field absent from the source is absent from the row. Nothing is inferred, modelled or filled in to make a row look complete.