OSF Preprints Scraper for Open Science Framework Preprints and Providers

OSF Preprints Scraper collects publicly available preprint and provider information from the Open Science Framework (OSF). Search preprints by title, optionally filter results by preprint server, or retrieve the list of available providers. Export the collected data to JSON, CSV, or Excel.

No login or application key is required.

Use Cases

• Literature searches: Find preprints with titles mentioning a research method or topic.
• Provider discovery: Export provider names and available submission-related attributes.
• Server-specific research: Filter preprint searches by a provider ID, such as psyarxiv.
• Research collaboration: Share collected records with a lab or research group.
• Scheduled monitoring: Repeat searches periodically to maintain an updated dataset.

Input Parameters

Parameter	Type	Default	Description
mode	string	search	Use search to collect preprints or providers to collect preprint provider records.
search	string	python	Title text used to filter preprints in search mode.
provider	string	Empty	Optional provider ID filter, such as psyarxiv. Applies in search mode.
maxItems	integer	100	Maximum number of successful records to save. Minimum: 1; maximum: 1,000.
timeoutSecs	integer	300	Overall time limit in seconds for starting new requests.
requestTimeoutSecs	integer	30	Timeout in seconds for an individual request.

Example Input

{
  "mode": "search",
  "search": "python",
  "provider": "psyarxiv",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}

To collect provider records instead, set "mode": "providers".

What Data Does This Actor Extract?

Each successful dataset record represents either a preprint or a preprint provider. The original record type is stored in recordType to maintain a consistent dataset schema. HTML in attribute text is converted to plain text, and Unicode characters in titles are preserved.

Example Output

{
  "id": "56ktq_v3",
  "recordType": "preprints",
  "itemTitle": "iTemplate2: An Open-Source Python Toolbox for Automatic Eye-Movement Analysis of Dynamic Facial Stimuli",
  "attributes": {
    "title": "iTemplate2: An Open-Source Python Toolbox for Automatic Eye-Movement Analysis of Dynamic Facial Stimuli",
    "description": "Template-based gaze-to-face registration enables standardized analysis of eye movements.",
    "is_published": true,
    "tags": [
      "eye-tracking",
      "face processing"
    ]
  },
  "relationships": {
    "provider": {
      "data": {
        "id": "psyarxiv",
        "type": "preprint-providers"
      }
    }
  },
  "links": {
    "self": "PREPRINT_RECORD_URL",
    "html": "PREPRINT_PAGE_URL",
    "preprint_doi": "DOI_URL"
  },
  "scrapedAt": "2026-10-12T00:00:00Z"
}

Output Fields

Field	Type	Description
id	string	Identifier of the preprint or provider.
recordType	string	Record type, such as preprints or preprint_providers.
itemTitle	string	Plain-text title of the preprint or provider name.
attributes	object	Record attributes, with HTML removed from text values.
relationships	object	Related records and associated metadata, including license information when available.
links	object	Available record, HTML page, and DOI links.
scrapedAt	string	UTC timestamp indicating when the record was saved.
error	string	Error details, present only on error records.
input	object	Input or request context associated with an error record.

How It Works

1. Reads the input and limits maxItems to a range of 1–1,000.
2. Collects preprint records or provider information according to the selected mode.
3. Retrieves records in pages, following available pagination links until the item limit is reached or no further page exists.
4. Waits approximately half a second between pages to limit request frequency.
5. Retries HTTP 429 and 5xx responses up to three additional times, using delays of 2, 4, and 8 seconds. A longer Retry-After value is respected.
6. Saves unique records by ID and records errors in the dataset when appropriate.

The provider listing may return a deprecation warning from the source. Records can still be saved when this warning occurs.

Limits and Terms

• The default record limit is 100, with a maximum of 1,000 successful records per run.
• The actor targets approximately two requests per second, with a page size of three records.
• A numeric source rate limit could not be confirmed during testing.
• Provider listings may take longer to return and may include a deprecation warning.
• A rejected request, invalid mode, or non-JSON response produces an error record rather than necessarily stopping the entire run.
• An empty title search produces an error record explaining that no records matched.
• Preprints may have individual licensing terms. Check the relevant license metadata before reusing or republishing content.

FAQ

Can I collect records from only one preprint server?

Yes. Set mode to search and specify a provider ID, such as psyarxiv or socarxiv, in the provider field.

Why is the record type stored as recordType?

The actor uses recordType to keep the dataset schema consistent while preserving the original type value.

What happens if a title search returns no results?

The dataset contains an error record explaining that no matching records were found, rather than preprint records.

How many records can one run collect?

The default limit is 100 successful records. You can increase maxItems to a maximum of 1,000.

What export formats are supported?

You can export the dataset as JSON, CSV, or Excel.

Trademark Notice

OSF Preprints and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of the data and does not imply affiliation, sponsorship, endorsement, or partnership.

Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with OSF Preprints or its owners. It collects publicly available information. Users are responsible for complying with the source's terms of use and applicable data, attribution, and licensing requirements.

Is Web Scraping Legal?

Collecting information that is already public is generally legal. Scraping is not the same thing as hacking, and there is no single law that bans it. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article Is web scraping legal?.