# Osf Preprints Scraper (`real-estate-api/osf-preprints-scraper`) Actor

OSF Preprints Scraper is a Python tool designed to extract preprint metadata, paper abstracts, author details, DOIs, and discipline tags from Open Science Framework (OSF) repositories into structured CSV or JSON formats for research and data analysis.

- **URL**: https://apify.com/real-estate-api/osf-preprints-scraper.md
- **Developed by:** [real-estate-api](https://apify.com/real-estate-api) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

![OSF Preprints Scraper for Open Science Framework Preprints and Providers](https://images.bigibyte.com/cdn-cgi/image/width=800,quality=80,format=auto/apify-actors-covers/osf-preprints-scraper-cover.png)

## OSF Preprints Scraper for Open Science Framework Preprints and Providers

OSF Preprints Scraper collects publicly available preprint and provider information from the Open Science Framework (OSF). Search preprints by title, optionally filter results by preprint server, or retrieve the list of available providers. Export the collected data to JSON, CSV, or Excel.

No login or application key is required.

### Use Cases

- **Literature searches:** Find preprints with titles mentioning a research method or topic.
- **Provider discovery:** Export provider names and available submission-related attributes.
- **Server-specific research:** Filter preprint searches by a provider ID, such as `psyarxiv`.
- **Research collaboration:** Share collected records with a lab or research group.
- **Scheduled monitoring:** Repeat searches periodically to maintain an updated dataset.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `search` | Use `search` to collect preprints or `providers` to collect preprint provider records. |
| `search` | string | `python` | Title text used to filter preprints in search mode. |
| `provider` | string | Empty | Optional provider ID filter, such as `psyarxiv`. Applies in search mode. |
| `maxItems` | integer | `100` | Maximum number of successful records to save. Minimum: 1; maximum: 1,000. |
| `timeoutSecs` | integer | `300` | Overall time limit in seconds for starting new requests. |
| `requestTimeoutSecs` | integer | `30` | Timeout in seconds for an individual request. |

### Example Input

```json
{
  "mode": "search",
  "search": "python",
  "provider": "psyarxiv",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}
```

To collect provider records instead, set `"mode": "providers"`.

### What Data Does This Actor Extract?

Each successful dataset record represents either a preprint or a preprint provider. The original record type is stored in `recordType` to maintain a consistent dataset schema. HTML in attribute text is converted to plain text, and Unicode characters in titles are preserved.

#### Example Output

```json
{
  "id": "56ktq_v3",
  "recordType": "preprints",
  "itemTitle": "iTemplate2: An Open-Source Python Toolbox for Automatic Eye-Movement Analysis of Dynamic Facial Stimuli",
  "attributes": {
    "title": "iTemplate2: An Open-Source Python Toolbox for Automatic Eye-Movement Analysis of Dynamic Facial Stimuli",
    "description": "Template-based gaze-to-face registration enables standardized analysis of eye movements.",
    "is_published": true,
    "tags": [
      "eye-tracking",
      "face processing"
    ]
  },
  "relationships": {
    "provider": {
      "data": {
        "id": "psyarxiv",
        "type": "preprint-providers"
      }
    }
  },
  "links": {
    "self": "PREPRINT_RECORD_URL",
    "html": "PREPRINT_PAGE_URL",
    "preprint_doi": "DOI_URL"
  },
  "scrapedAt": "2026-10-12T00:00:00Z"
}
```

#### Output Fields

| Field | Type | Description |
|---|---|---|
| `id` | string | Identifier of the preprint or provider. |
| `recordType` | string | Record type, such as `preprints` or `preprint_providers`. |
| `itemTitle` | string | Plain-text title of the preprint or provider name. |
| `attributes` | object | Record attributes, with HTML removed from text values. |
| `relationships` | object | Related records and associated metadata, including license information when available. |
| `links` | object | Available record, HTML page, and DOI links. |
| `scrapedAt` | string | UTC timestamp indicating when the record was saved. |
| `error` | string | Error details, present only on error records. |
| `input` | object | Input or request context associated with an error record. |

### How It Works

1. Reads the input and limits `maxItems` to a range of 1–1,000.
2. Collects preprint records or provider information according to the selected mode.
3. Retrieves records in pages, following available pagination links until the item limit is reached or no further page exists.
4. Waits approximately half a second between pages to limit request frequency.
5. Retries HTTP 429 and 5xx responses up to three additional times, using delays of 2, 4, and 8 seconds. A longer `Retry-After` value is respected.
6. Saves unique records by ID and records errors in the dataset when appropriate.

The provider listing may return a deprecation warning from the source. Records can still be saved when this warning occurs.

### Limits and Terms

- The default record limit is 100, with a maximum of 1,000 successful records per run.
- The actor targets approximately two requests per second, with a page size of three records.
- A numeric source rate limit could not be confirmed during testing.
- Provider listings may take longer to return and may include a deprecation warning.
- A rejected request, invalid mode, or non-JSON response produces an error record rather than necessarily stopping the entire run.
- An empty title search produces an error record explaining that no records matched.
- Preprints may have individual licensing terms. Check the relevant license metadata before reusing or republishing content.

### FAQ

#### Can I collect records from only one preprint server?

Yes. Set `mode` to `search` and specify a provider ID, such as `psyarxiv` or `socarxiv`, in the `provider` field.

#### Why is the record type stored as `recordType`?

The actor uses `recordType` to keep the dataset schema consistent while preserving the original type value.

#### What happens if a title search returns no results?

The dataset contains an error record explaining that no matching records were found, rather than preprint records.

#### How many records can one run collect?

The default limit is 100 successful records. You can increase `maxItems` to a maximum of 1,000.

#### What export formats are supported?

You can export the dataset as JSON, CSV, or Excel.

### Trademark Notice

OSF Preprints and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of the data and does not imply affiliation, sponsorship, endorsement, or partnership.

### Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with OSF Preprints or its owners. It collects publicly available information. Users are responsible for complying with the source's terms of use and applicable data, attribution, and licensing requirements.

### Is Web Scraping Legal?

Collecting information that is already public is generally legal. Scraping is not the same thing as hacking, and there is no single law that bans it. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/).

# Actor input Schema

## `mode` (type: `string`):

Search reads public preprints from /v2/preprints/. Providers lists preprint servers from /v2/preprint_providers/.

## `search` (type: `string`):

Title text sent as filter\[title] on the public preprints endpoint. Used in search mode.

## `provider` (type: `string`):

Optional preprint provider id, such as psyarxiv or socarxiv. Sent as filter\[provider] in search mode. Leave blank to search every provider.

## `maxItems` (type: `integer`):

Maximum successful rows to save in one run. Values above 1000 are clamped to 1000.

## `timeoutSecs` (type: `integer`):

Stop scheduling new requests after this many seconds. Rows already saved are kept.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds for each call to the OSF API.

## `proxyConfiguration` (type: `object`):

Optional proxy. Default is no proxy. The public OSF API does not need a proxy.

## Actor input object example

```json
{
  "mode": "search",
  "search": "python",
  "provider": "psyarxiv",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetUrl` (type: `string`):

Default dataset for this run.

## `itemsUrl` (type: `string`):

Dataset items for this run.

## `overview` (type: `string`):

Table of OSF preprint or provider rows from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "search",
    "search": "python",
    "provider": "",
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("real-estate-api/osf-preprints-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "search",
    "search": "python",
    "provider": "",
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("real-estate-api/osf-preprints-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "search",
  "search": "python",
  "provider": "",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call real-estate-api/osf-preprints-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,real-estate-api/osf-preprints-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Enxg7IVest5rpYgN3/builds/BbkYWryvARtBahWMM/openapi.json