# bioRxiv Scraper · Preprints, Authors, Categories & Versions (`reapx/biorxiv-scraper`) Actor Scrape bioRxiv and medRxiv research preprints, authors, categories, versions, licensing, and abstracts via public REST API. Pay-per-event pricing per preprint record. - **URL**: https://apify.com/reapx/biorxiv-scraper.md - **Developed by:** [Tarek Etman](https://apify.com/reapx) (community) - **Categories:** Developer tools, Business, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.30 / 1,000 record returneds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![reapX — public sources in, addressable records out](https://reapx.dev/reapx.gif) ## bioRxiv Scraper · Preprints, Authors, Categories & Versions Extract preprints, authors, categories, versions, licensing, and abstracts from bioRxiv and medRxiv repositories using the **bioRxiv scraper**. Query scientific literature across biological, medical, and health sciences with flexible date windows, category filters, and keyword search via direct HTTP API access. > Maintained by **reapX**. Every row cites the Apify run that produced it — nothing is > inferred, modelled or filled in, and a field absent from the source is absent from the row. > The extracted archive for this source is browsable at > [reapx.dev/data/biorxiv-scraper/](https://reapx.dev/data/biorxiv-scraper/) and mirrored as an open dataset on > [Hugging Face](https://huggingface.co/datasets/reapxdev/biorxiv-scraper) and > [Kaggle](https://www.kaggle.com/datasets/reapxdev/biorxiv-scraper). Questions: reapxdev@proton.me *** ### Features & Capabilities - **Dual Repository Support**: Seamlessly fetch manuscripts from both **bioRxiv** (biological sciences) and **medRxiv** (medical, clinical, and health sciences). - **Comprehensive Metadata Extraction**: Capture paper DOIs, normalized paper IDs, full titles, authors, corresponding authors, institutional affiliations, posting dates, manuscript versions, subject categories, article types, open-access licenses, funding statements, and full abstracts. - **Granular Date Window Filtering**: Target specific publication date ranges using standard `YYYY-MM-DD` start and end dates. - **Subject Category Filtering**: Isolate research in neuroscience, bioinformatics, genetics, cell biology, immunology, epidemiology, infectious diseases, clinical oncology, health informatics, and over 60 specific subject categories. - **Keyword & Author Search**: Filter preprints by specific keywords in titles/abstracts (e.g. `CRISPR`, `deep learning`, `mRNA vaccine`, `GWAS`) or author names. - **Schema-Compliant Entity Routing**: Every row includes canonical `doi` and slugified `paperId` fields matching standard entity page resolution requirements. - **Pay-Per-Event (PPE) Pricing**: Pay only per successful preprint record written to the default dataset. Rate-limited requests, 429 backoffs, and empty searches are never billed. - **High Throughput & Reliability**: Built using light HTTP client requests with automatic exponential backoff on 429 and 5xx responses. *** ### ⬇️ Input Configure the **bioRxiv scraper** using the parameters below. All parameters carry working defaults or prefilled values for immediate execution. | Input Field | Type | Required | Default / Prefill | Description | | :--- | :--- | :--- | :--- | :--- | | `server` | String | No | `biorxiv` | Target repository server: `biorxiv` for biological sciences or `medrxiv` for medical sciences. | | `fromDate` | String | No | `2024-01-01` | Start date for posting date window (`YYYY-MM-DD`). | | `toDate` | String | No | `2024-01-05` | End date for posting date window (`YYYY-MM-DD`). | | `category` | String | No | *Empty* | Subject category filter (e.g. `neuroscience`, `bioinformatics`, `epidemiology`). | | `term` | String | No | *Empty* | Keyword phrase search filter matching paper title and abstract text. | | `author` | String | No | *Empty* | Author name filter string matching primary or co-authors. | | `maxPapers` | Integer | Yes | `50` | Maximum number of preprint records to extract (1 to 10,000). | #### Example Input Configurations ##### 1. Basic bioRxiv Neuroscience Preprints ```json { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "category": "neuroscience", "maxPapers": 50 } ``` ##### 2. medRxiv Epidemiology & Infectious Diseases Search ```json { "server": "medrxiv", "fromDate": "2024-01-01", "toDate": "2024-01-10", "category": "epidemiology", "term": "outbreak", "maxPapers": 100 } ``` ##### 3. CRISPR & Gene Editing Research Across Repositories ```json { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-15", "term": "CRISPR", "maxPapers": 100 } ``` *** ### ⬆️ Output All extracted records are pushed directly to the run's default dataset in real time. #### Dataset Field Schema | Field Name | Type | Description | Example | | :--- | :--- | :--- | :--- | | `doi` | String | Digital Object Identifier assigned to the preprint. | `10.1101/2023.12.30.573731` | | `paperId` | String | Normalized identifier with colons replaced by hyphens for entity page addressing. | `10.1101-2023.12.30.573731` | | `title` | String | Full scientific manuscript title. | `KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents` | | `authors` | String | Semicolon-separated list of listed authors. | `Sinha, A. K.; Lee, C.; Holt, J. C.` | | `authorCorresponding` | String | Full name of corresponding author. | `Joseph C Holt` | | `authorCorrespondingInstitution` | String | Institutional affiliation of corresponding author. | `Dept. of Pharmacology, Univ. of Rochester` | | `date` | String | Manuscript posting date (`YYYY-MM-DD`). | `2024-01-01` | | `version` | String | Preprint manuscript version iteration. | `1` | | `type` | String | Article submission classification type. | `new results` | | `license` | String | Open-access licensing code. | `cc_by_nc_nd` | | `category` | String | Subject area research category. | `neuroscience` | | `jatsxml` | String | Direct URL link to full JATS XML source file. | `https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml` | | `abstract` | String | Complete abstract text. | `Primary vestibular afferents transmit information...` | | `funder` | String | Funding agency or grant information. | `NIH R01 DC00429` | | `published` | String | Journal DOI reference if peer-reviewed and published. | `10.1016/j.neuron.2024.03.012` | | `server` | String | Repository server name (`bioRxiv` or `medRxiv`). | `bioRxiv` | | `url` | String | Direct web URL to preprint webpage. | `https://www.biorxiv.org/content/10.1101/2023.12.30.573731v1` | #### Sample JSON Output Record ```json { "doi": "10.1101/2023.12.30.573731", "paperId": "10.1101-2023.12.30.573731", "title": "KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents in mammals", "authors": "Sinha, A. K.; Lee, C.; Holt, J. C.", "authorCorresponding": "Joseph C Holt", "authorCorrespondingInstitution": "Dept. of Pharmacology and Physiology, Univ. of Rochester, Rochester, New York 14642", "date": "2024-01-01", "version": "1", "type": "new results", "license": "cc_no", "category": "neuroscience", "jatsxml": "https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml", "abstract": "Primary vestibular afferents transmit information from hair cells about head position and movement to the CNS, which is critical for maintaining balance, gaze stability and spatial navigation...", "funder": "NA", "published": "NA", "server": "bioRxiv", "url": "https://www.biorxiv.org/content/10.1101/2023.12.30.573731" } ``` #### Dataset Named Views The dataset schema includes three pre-configured views tailored for different analytical workflows: 1. **Overview (`overview`)**: Columns `doi`, `paperId`, `title`, `server`, `category`, `date`, `version`, `url`. Ideal for rapid screening. 2. **Authors & Affiliations (`authors_and_affiliations`)**: Columns `paperId`, `title`, `authors`, `authorCorresponding`, `authorCorrespondingInstitution`, `funder`. Focused on bibliometric and institutional research. 3. **Metadata & Licensing (`metadata_and_licensing`)**: Columns `paperId`, `title`, `doi`, `server`, `category`, `type`, `license`, `published`, `jatsxml`. Built for open access tracking and XML corpus pipelines. *** ### How it works The **bioRxiv scraper** connects directly to the official bioRxiv details REST API (`https://api.biorxiv.org/details/{server}/{from}/{to}/{cursor}`). 1. **API Handshake**: Validates input parameters, target server (`biorxiv` or `medrxiv`), and date window. 2. **Paginated Retrieval**: Issues lightweight HTTP GET requests using cursor pagination (30 records per request window). 3. **Client-side Filtering**: Filters records in memory by subject category, author name, or title/abstract keywords if specified. 4. **Resilience & Backoff**: Automatically handles rate limits (HTTP 429) and server glitches (HTTP 5xx) with exponential backoff (2s, 4s, 8s, 16s) up to 5 retries. 5. **Pay-Per-Event Charging**: Charges exactly `$0.002` per valid preprint record via `Actor.charge()` immediately before dataset push. 6. **Streaming Dataset Output**: Emits items continuously to the run's default dataset so no data is lost in the event of run interruption. *** ### Pay-Per-Event Pricing & Volume Discounts This actor uses **Pay-Per-Event (PPE)** pricing. You are charged only for actual preprint records returned and stored in your dataset. - **Base Price**: `$0.002` per preprint record. - **Platform Usage**: Included. Compute usage is absorbed into the event price. - **Tiered Volume Discounts**: Automatically applied based on your monthly Apify plan tier: - **FREE**: `$0.0020` / record - **BRONZE**: `$0.0018` / record (10% discount) - **SILVER**: `$0.0015` / record (25% discount) - **GOLD**: `$0.0013` / record (35% discount) - **PLATINUM**: `$0.0010` / record (50% discount) - **DIAMOND**: `$0.0008` / record (60% discount) *** ### Use Cases - **Biomedical Literature Mining**: Track cutting-edge preprints across cancer research, genomics, neurobiology, and immunology before formal journal peer review. - **Epidemiology & Outbreak Surveillance**: Monitor real-time public health studies, epidemiological models, and infectious disease findings on medRxiv. - **AI & LLM Training Datasets**: Aggregate high-quality scientific abstracts and JATS XML links for fine-tuning domain-specific language models. - **Competitive Intelligence**: Follow research output from academic labs and biotechnology companies by tracking author affiliations and funding grants. *** ### ❓ FAQ ##### Q: Do I need an API key or account to use bioRxiv Scraper? No. The scraper accesses the open bioRxiv / medRxiv public API directly over HTTP. No external account or API key is required. ##### Q: Which preprint servers are supported? Both `biorxiv` (bioRxiv for biological sciences) and `medrxiv` (medRxiv for medical and health sciences) are fully supported. ##### Q: How are date bounds formatted? Use standard `YYYY-MM-DD` dates (e.g. `2024-01-01`). The bioRxiv API filters manuscripts posted within the date window. ##### Q: What happens if a search returns zero results? If no preprints match your search filters or date range, the actor completes cleanly with 0 rows emitted and **charges 0 USD**. ##### Q: Can I export results to CSV, Excel, or JSON? Yes. All results saved to the default dataset can be downloaded from the Apify Console or API in JSON, CSV, Excel, XML, or HTML table format. *** ### 💬 Your feedback Have suggestions, feature requests, or bug reports? We continuously improve our scrapers based on user feedback. Reach out directly via email at **reapxdev@proton.me** or visit [reapx.dev](https://reapx.dev). *** *Disclaimer: Unofficial - not affiliated with bioRxiv or medRxiv. Collects public data only. reapx. Contact reapxdev@proton.me.* ### 🧪 Example input A real, runnable configuration — this is an actual input this Actor has run with. ```json { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "maxPapers": 50, "category": "neuroscience" } ``` ### 📄 Sample output One real row from a real run of this Actor, unedited. ```json { "doi": "10.1101/2023.12.30.573731", "paperId": "10.1101/2023.12.30.573731", "title": "KCNQ2/3 regulates efferent mediated slow excitation of vestibular afferents in mammals", "authors": "Sinha, A. K.; Lee, C.; Holt, J. C.", "authorCorresponding": "Joseph C Holt", "authorCorrespondingInstitution": "Dept. of Pharmacology and Physiology, Univ. of Rochester, Rochester, New York 14642", "date": "2024-01-01", "version": "1", "type": "new results", "license": "cc_no", "category": "neuroscience", "jatsxml": "https://www.biorxiv.org/content/early/2024/01/01/2023.12.30.573731.source.xml", "abstract": "Primary vestibular afferents transmit information from hair cells about head position and movement to the CNS, which is critical for maintaining balance, gaze stability and spatial navigation. The CNS, in turn, modulates hair cells and afferents via the efferent vestibular system (EVS) and its activation of several cholinergic signaling mechanisms. Electrical stimulation of EVS neurons gives rise to three kinetically- and mechanistically-distinct afferent responses including a slow excitation, a fast excitation, and a fast inhibition. EVS-mediated slow excitation is attributed to odd-numbered muscarinic acetylcholine receptors (mAChRs) on the afferent whose activation leads to the closure of a potassium conductance and increased afferent discharge.\n\nLikely effector candidates include low-threshold, voltage-gated potassium channels belonging to the KCNQ (Kv7.X) family, which are involved in neuronal excitability across the nervous system and are subject to mAChR modulation. Specifically, KCNQ2/3 heteromeric channels may be the molecular correlates for the M-current, a potassium current that is blocked following the activation of odd-numbered mAChRs. To this end, multiple members of the KCNQ channel family, including KCNQ2 and KCNQ3, are localized to several microdomains within vestibular afferent endings, where they influence afferent excitability and could be targeted by EVS neurons. Additionally, the relative expression of KCNQ subunits appears to vary across the sensory epithelia and among different afferent types. However, it is unclear which KCNQ channel subunits are targeted by mAChR activation and whether that also varies among different afferent classes. Here we show that EVS-mediated slow excitation is blocked and enhanced by the non-selective KCNQ channel blocker XE991 and opener retigabine, respectively. Using KCNQ subunit-selective drugs, we observed that a KCNQ2 blocker blocks the slow response in irregular afferents, while a KCNQ2/3 opener enhances slow responses in regular afferents. The KCNQ2 blockers did not appear to affect resting afferent discharge rates, while KCNQ2/3 or KCNQ2/4 openers decreased afferent excitability. Here, we show pharmacological evidence that KCNQ2/3 subunits are likely targeted by mAChR activation in mammalian vestibular afferents. Additionally, we show that KCNQ3 KO mice have altered resting discharge rate as well as EVS-mediated slow response. These data together suggest that KCNQ channels play a role in slow response and discharge rate of vestibular afferents, which can be modulated ``` ### ⚠️ Run outcomes and error handling This Actor reports what happened in the run's **status message**, and it always keeps whatever it collected. These are the outcomes you can get and what each one means. | Outcome | What it means | |---|---| | **Success** | Rows were returned and you were charged `record-returned` at $0.002 per row. | | **No matches** | The source returned nothing for your filters. **Nothing is charged.** Widen the date window or drop a filter. | #### What is guaranteed either way - **Every row is pushed as it is built**, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not. - **A field absent from the source is absent from the row.** Nothing is inferred, modelled or filled in to make a row look complete. # Actor input Schema ## `server` (type: `string`): Select the preprint repository server to query (biorxiv or medrxiv). Prefilled with biorxiv.

Choosing biorxiv extracts biological sciences research while medrxiv extracts medical and health sciences manuscripts. Leaving this unselected targets bioRxiv by default. ## `fromDate` (type: `string`): Start date of the posting date window in YYYY-MM-DD format (e.g. 2024-01-01). Prefilled with 2024-01-01.

Leaving this field empty defaults to searching recent preprints without a lower bound. Setting a start date narrows the query timeline and reduces total execution cost. ## `toDate` (type: `string`): End date of the posting date window in YYYY-MM-DD format (e.g. 2024-01-05). Prefilled with 2024-01-05.

Leaving this field empty includes all papers posted up to today. Constraining the end date limits the result set and ensures faster actor execution times. ## `category` (type: `string`): Filter manuscripts by research subject category (e.g. neuroscience, genetics, bioinformatics, epidemiology, or infectious diseases).

Leave empty to extract preprints across all subject areas. Specifying a category isolates relevant discipline papers and speeds up data delivery. ## `term` (type: `string`): Keyword or phrase search filter to match against manuscript titles and abstract text (e.g. CRISPR, machine learning, or mRNA vaccine).

Leave empty to return all preprints in the date window. Adding search terms filters out irrelevant manuscripts and saves event charges. ## `author` (type: `string`): Author name filter string to match against primary or co-authors listed on preprints (e.g. Sinha or Holt).

Leave this field empty to include papers from all research groups. Filtering by author isolates publications from specific laboratory investigators. ## `maxPapers` (type: `integer`): Maximum number of preprint records to extract and save into the default dataset (1 to 10000). Prefilled with 50.

Higher limits result in longer execution times and higher total event charges. Setting a smaller limit caps the run cost.

The higher the number, the longer the run takes and the more you are charged, because you pay per row returned. ## Actor input object example ```json { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "maxPapers": 50 } ``` # Actor output Schema ## `results` (type: `string`): Every bioRxiv or medRxiv preprint record extracted by this run, one row per paper, in the default dataset. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "maxPapers": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("reapx/biorxiv-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "maxPapers": 50, } # Run the Actor and wait for it to finish run = client.actor("reapx/biorxiv-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "server": "biorxiv", "fromDate": "2024-01-01", "toDate": "2024-01-05", "maxPapers": 50 }' | apify call reapx/biorxiv-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,reapx/biorxiv-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/vmaaF18G01QQluhbQ/builds/Dv5swfhia4hMt89ga/openapi.json