# Biorxiv Medrxiv Preprints Scraper (`zerobreak/biorxiv-medrxiv-preprints-scraper`) Actor

The BioRxiv & MedRxiv Preprints Scraper automatically collects research papers and metadata from public biology and medical preprint servers. It helps researchers, journalists, and analysts easily gather early-stage scientific findings and export them into structured formats like CSV and JSON.

- **URL**: https://apify.com/zerobreak/biorxiv-medrxiv-preprints-scraper.md
- **Developed by:** [ZeroBreak](https://apify.com/zerobreak) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## BioRxiv/medRxiv Preprints Scraper: Preprints by Date Range, No API Key

<p align="center"><img src="https://imgur.com/kclFAbJ.png" alt="bioRxiv medRxiv Preprints Scraper" width="800" /></p>

bioRxiv/medRxiv Preprints Scraper collects publicly available preprint metadata from bioRxiv and medRxiv using their public details API. No login or API key is required.

Collect preprints within a specified date range or retrieve individual records using their DOIs. Export structured results as JSON, CSV, or Excel for literature reviews, scientific research, and data analysis.

Each output row represents a preprint version and includes its title, authors, corresponding author, DOI, posting date, license, subject category, abstract, and additional metadata. Text fields are converted to plain text where appropriate, Unicode characters are preserved, and duplicate DOI-version combinations are removed.

### Features

- **Date-range collection:** Retrieve preprints posted within a specified date window.
- **DOI lookup:** Retrieve individual preprint records using DOI identifiers.
- **Cross-server fallback:** Retry a DOI lookup on the other server when no matching record is found on the selected server.
- **Structured metadata:** Collect titles, authors, institutions, DOIs, versions, licenses, categories, abstracts, and publication information.
- **Text cleaning:** Remove HTML from applicable text fields while preserving Unicode characters.
- **Duplicate detection:** Deduplicate records using DOI and version.
- **Automatic retries:** Retry HTTP 429 responses, server errors, and timeouts with exponential backoff.
- **Error handling:** Save errors as separate dataset rows instead of terminating the entire run.
- **Configurable limits:** Control the maximum number of records and request timeouts.

### Use Cases

- **Literature reviews:** Collect recent preprints in a specific research area.
- **Scientific research:** Build structured preprint datasets for analysis and referencing.
- **DOI-based retrieval:** Retrieve known preprints without collecting an entire date range.
- **Research monitoring:** Schedule recurring runs to identify newly posted preprints.
- **License tracking:** Preserve license information associated with each record.
- **Data pipelines:** Export metadata and abstracts for Python, Pandas, research databases, and reporting workflows.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `server` | string | `biorxiv` | Select `biorxiv` or `medrxiv`. If a DOI is not found on the selected server, the actor retries on the other server. |
| `fromDate` | string | `2026-10-01` | Start date in `YYYY-MM-DD` format. Used when no DOIs are supplied. |
| `toDate` | string | `2026-10-10` | End date in `YYYY-MM-DD` format. Must be on or after `fromDate`. |
| `dois` | array | Empty | Optional list of DOIs, with one DOI per item. When populated, the date range is ignored. |
| `maxItems` | integer | 100 | Maximum number of successful records to save. Maximum: 1000. |
| `timeoutSecs` | integer | 300 | Overall runtime limit in seconds. Maximum: 3600. |
| `requestTimeoutSecs` | integer | 30 | Maximum wait time for an individual request, in seconds. Maximum: 120. |

### Example Input: Date Range

```json
{
  "server": "biorxiv",
  "fromDate": "2026-10-01",
  "toDate": "2026-10-10",
  "dois": [],
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}
```

### Example Input: Specific DOIs

When the `dois` array contains at least one entry, the date range is ignored.

```json
{
  "server": "biorxiv",
  "dois": [
    "10.1101/2020.09.09.20191205"
  ],
  "maxItems": 100
}
```

### What Data Does This Scraper Extract?

Each successful dataset item represents one preprint version. The API title is stored in `paperTitle`, while the API's record type is stored in `preprintType` to maintain the dataset's field naming conventions.

HTML is removed from applicable text fields, Unicode characters in names are preserved, and the original license code is retained.

#### Example Output

```json
{
  "paperTitle": "The Role of the Left Inferior Frontal Gyrus in Resolving Immediate and Carryover Semantic Interference During Picture Naming",
  "authors": "Harvey, D. Y.; Stoll, H.; Hamilton, R. H.",
  "author_corresponding": "Denise Y Harvey",
  "author_corresponding_institution": "University of Pennsylvania",
  "doi": "10.64898/2026.09.28.755022",
  "date": "2026-10-01",
  "version": "2",
  "preprintType": "new results",
  "license": "cc_no",
  "category": "neuroscience",
  "jatsxml": "JATS_XML_URL",
  "abstract": "Naming pictures from the same semantic category can interfere with subsequent word retrieval.",
  "published": "NA",
  "server": "bioRxiv",
  "sourceUrl": "API_REQUEST_URL",
  "scrapedAt": "2026-10-12T00:00:00+00:00"
}
```

### Output Fields

| Field | Type | Description |
|---|---|---|
| `paperTitle` | string | Preprint title returned by the API. |
| `authors` | string | List of authors associated with the preprint. |
| `author_corresponding` | string | Name of the corresponding author, when available. |
| `author_corresponding_institution` | string | Institution associated with the corresponding author. |
| `doi` | string | Digital Object Identifier for the preprint. |
| `date` | string | Preprint posting date. |
| `version` | string | Preprint version number. |
| `preprintType` | string | Record type returned by the API, such as `new results`. |
| `license` | string | License code associated with the record. |
| `category` | string | Research subject category. |
| `jatsxml` | string | Reference to the preprint's JATS XML resource. |
| `abstract` | string | Abstract converted to plain text. |
| `published` | string | Published article DOI, when available, or `NA`. |
| `server` | string | Server associated with the record: bioRxiv or medRxiv. |
| `sourceUrl` | string | Request URL associated with the returned record. |
| `scrapedAt` | string | UTC timestamp indicating when the record was saved. |
| `error` | string | Error description, included only in error rows. |
| `input` | object | Original actor input copied into error rows. |
| `httpStatus` | integer | HTTP status code, when available for an error response. |

An invalid date, invalid DOI, empty result, or non-JSON response may produce an error row. The actor continues processing other requests whenever possible.

### How It Works

1. Validates the selected server, date range, and record limits.
2. If no DOIs are provided, requests preprint records from the selected server for the specified date range.
3. If DOIs are provided, retrieves records individually using the supplied identifiers.
4. If a DOI is not found on the selected server, retries the lookup once on the other server.
5. Follows cursor-based pagination until the available records are processed or a configured limit is reached.
6. Waits approximately one second between requests to avoid excessive traffic.
7. Retries HTTP 429 responses, HTTP 5xx errors, and timeouts up to three times, using delays of 2, 4, and 8 seconds. Longer `Retry-After` delays are respected when provided.
8. Removes duplicate records with the same DOI and version.
9. Saves successful records and error rows to the output dataset.

### Troubleshooting

#### The Dataset Contains Only an Error Row

The input may not match any available records, or one of the supplied values may be invalid. Check the `error` field for details.

A message indicating that no posts were found may mean the DOI or date range returned no matching records.

#### The Date Range Is Invalid

Both dates must use `YYYY-MM-DD` format. Ensure that `fromDate` is on or before `toDate`.

#### Fewer Records Were Collected Than Expected

Check `maxItems`, which defaults to 100 and cannot exceed 1000. Also check `timeoutSecs`, which prevents new requests from starting after the configured runtime limit. Previously saved rows remain in the dataset.

#### Requests Are Blocked or Time Out

Increase `requestTimeoutSecs` when the service responds slowly. If requests are being blocked, review the service's usage policies and adjust the collection frequency.

#### A DOI Returns No Results

The actor retries the lookup on the other server once when the selected server returns no matching record. Confirm that the DOI is valid and belongs to bioRxiv or medRxiv.

#### The Dataset Contains Duplicate-Looking Records

Different versions of the same preprint may legitimately appear as separate records. Deduplication uses the combination of DOI and version rather than DOI alone.

### Limits and Data Usage

- The public details API provides access to preprint metadata without requiring an API key.
- The supplied API information does not specify a numeric request-rate limit.
- The actor sends approximately one request per second.
- The maximum number of successful records per run is 1000.
- The maximum overall runtime is 3600 seconds.
- Individual request timeouts are limited to 120 seconds.
- Each preprint record carries its own license information. Preserve the license when reusing abstracts or other content.
- License conditions may vary. Check the applicable license before redistributing abstracts, full text, or other copyrighted material.

### Frequently Asked Questions

#### Does this scraper require an API key?

No. It uses the public details API and does not require login credentials or an API key.

#### Can I search both servers in one date range?

A date-range search uses the selected server. DOI lookups that return no matching record on that server are retried on the other server.

#### Why is the title field called `paperTitle`?

The scraper uses `paperTitle` for the title and `preprintType` for the record type to maintain its dataset schema.

#### What happens if the date range returns no records?

The run may finish with an error row describing the empty result. The API's message is retained when available.

#### How many records can one run save?

The default limit is 100 successful records, with a maximum of 1000.

#### Can I export results to CSV, JSON, and Excel?

Yes. Export the dataset in these formats for literature reviews, statistical analysis, research databases, and reporting.

#### Can I collect new preprints automatically?

Yes. Schedule recurring runs and use DOI-version deduplication to maintain a growing dataset over time.

### Pricing

The actor is free to run. Apify platform usage is billed separately according to the applicable plan.

### Integrations

Export the dataset and incorporate it into research workflows using compatible spreadsheet tools, automation platforms, databases, and data-processing pipelines.

### Trademark Notice

bioRxiv, medRxiv, and related names, logos, and trademarks belong to their respective owners. They are referenced only to identify the data sources and do not imply affiliation, sponsorship, endorsement, or partnership.

### Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with bioRxiv, medRxiv, or their owners.

Data is collected from publicly accessible sources. Users are responsible for complying with applicable source terms, rate limits, licensing conditions, and attribution requirements.

### Is Web Scraping Legal?

Collecting publicly available information through a public API is generally permitted, but the legality and conditions of use depend on the source's terms, applicable laws, data licenses, and intended purpose.

- **Personal data:** Author names and institutional affiliations may constitute personal information in some contexts. Follow applicable privacy requirements when collecting, storing, or redistributing such data.
- **Copyright and licensing:** Preprint abstracts, manuscripts, and related content may be subject to copyright or specific license terms. Check the license associated with each record before redistributing its content.
- **Terms of use:** Respect the source's usage policies, request limits, and attribution requirements.
- **Legal advice:** This information is provided for general educational purposes and does not constitute legal advice. Consult a qualified professional if you need guidance on a specific use case.

# Actor input Schema

## `server` (type: `string`):

Which public server to query first: biorxiv or medrxiv. DOI lookups that miss on this server are tried on the other one.

## `fromDate` (type: `string`):

Start of the posting interval, YYYY-MM-DD. Used when dois is empty. Sent in the path of /details/{server}/{from}/{to}/{cursor}.

## `toDate` (type: `string`):

End of the posting interval, YYYY-MM-DD. Used when dois is empty. Must be the same day as fromDate or later.

## `dois` (type: `array`):

Optional DOI list. When this has at least one DOI, the actor calls /details/{server}/{doi} and skips the date range. One DOI per line.

## `maxItems` (type: `integer`):

Maximum successful preprint rows to save in one run. Values above 1000 are clamped to 1000.

## `timeoutSecs` (type: `integer`):

Stop scheduling new requests after this many seconds. Rows already saved are kept.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds for each call to api.biorxiv.org.

## `proxyConfiguration` (type: `object`):

Optional proxy. Default is no proxy. The public bioRxiv details API does not need a proxy.

## Actor input object example

```json
{
  "server": "biorxiv",
  "fromDate": "2026-10-01",
  "toDate": "2026-10-10",
  "dois": [],
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetUrl` (type: `string`):

Default dataset for this run.

## `itemsUrl` (type: `string`):

Dataset items for this run.

## `overview` (type: `string`):

Table of preprint rows from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "server": "biorxiv",
    "fromDate": "2026-10-01",
    "toDate": "2026-10-10",
    "dois": [],
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("zerobreak/biorxiv-medrxiv-preprints-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "server": "biorxiv",
    "fromDate": "2026-10-01",
    "toDate": "2026-10-10",
    "dois": [],
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("zerobreak/biorxiv-medrxiv-preprints-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "server": "biorxiv",
  "fromDate": "2026-10-01",
  "toDate": "2026-10-10",
  "dois": [],
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call zerobreak/biorxiv-medrxiv-preprints-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,zerobreak/biorxiv-medrxiv-preprints-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ufUubxva9tnHhh0Ya/builds/1CwG5w1ohgSWfNS8y/openapi.json