OSF Open Science Framework Scraper

Scrape open-science projects (nodes), preprints, and preregistrations (registrations) from OSF, the Open Science Framework — with titles, descriptions, DOIs, subjects, tags, licenses, dates, and (optionally) contributor names. No API key required.

Built on the keyless read tier of the OSF API v2 (https://api.osf.io/v2), which returns data in JSON:API format. Runs at roughly ~$0.5 per 1,000 records.

What is OSF (and how is it different from Figshare / Zenodo)?

OSF is the free scholarly infrastructure run by the Center for Open Science. It is not a file-repository like Figshare or Zenodo — it is a research-workflow hub:

Platform	Core object	What you get
OSF (this actor)	Projects, preregistrations, preprints	The living research project: hypotheses registered before data collection, project structure, preprints, contributors.
Figshare (ninhothedev/figshare-scraper)	Published datasets & figures	Individual research outputs with DOIs.
Zenodo (ninhothedev/zenodo-scraper)	Archived datasets, software, publications	CERN-hosted long-term archive records.

In short: OSF = the open-science process (projects + preregistrations + preprints); Figshare/Zenodo = the archived artifacts. If you want to track what researchers said they would do before they did it (preregistration), OSF is the source.

JSON:API note

OSF speaks JSON:API: every response data is a list of resource objects shaped {"id", "type", "attributes", "relationships", "links"}. The real fields live under attributes, the web link under links.html, and related collections (like contributors) under relationships. This actor flattens all of that into clean, one-level records for you.

Modes

Pick with the mode input:

• preprints (default) — OSF-hosted preprints (PsyArXiv, SocArXiv, etc.) with doi, subjects, publication dates.
• nodes — public research projects; optionally enriched with contributor names.
• registrations — preregistrations, with the registration schema and registration date.

Input

Field	Type	Default	Description
mode	select	preprints	nodes, preprints, or registrations.
searchQuery	string	—	Title keyword filter (OSF filter[title]). E.g. climate.
enrichContributors	boolean	false	Nodes only. One extra API call per project to add contributor names.
maxItems	integer	200	1–3000.

OSF has no keyless full-text /search/ endpoint (it returns 404 without a token), so searchQuery filters on the title field via filter[title].

Output examples

Preprint

{
  "type": "preprint",
  "preprint_id": "9vt5b_v2",
  "title": "Voice Can Help and Hurt AI Empathy",
  "doi": "10.31234/osf.io/9vt5b_v2",
  "is_published": true,
  "date_published": "2025-03-01T00:00:00+00:00",
  "subjects": ["Social and Behavioral Sciences", "Affect and Emotion Regulation"],
  "html_url": "https://osf.io/preprints/psyarxiv/9vt5b_v2/",
  "source": "osf"
}

Node (project)

{
  "type": "osf_node",
  "node_id": "jyq3r",
  "title": "How much evidence for climate-driven dengue transmission survives the analyst?",
  "category": "project",
  "is_public": true,
  "is_registration": false,
  "tags": ["climate"],
  "contributors": ["Jane Doe"],
  "contributor_count": 1,
  "html_url": "https://osf.io/jyq3r/",
  "source": "osf"
}

Registration

{
  "type": "registration",
  "registration_id": "abc12",
  "title": "Preregistration of ...",
  "registration_schema": "OSF Preregistration",
  "date_registered": "2026-08-11T02:00:43+00:00",
  "is_public": true,
  "source": "osf"
}

Use cases

• Meta-research — study how open science is practiced at scale.
• Preregistration tracking — monitor new preregistrations in a field.
• Preprint discovery — surface fresh preprints with DOIs and subjects.
• Open science monitoring — watch projects/registrations by keyword.

Rate limits (observed)

The keyless read tier is generous. In testing, a burst of 30 rapid requests returned zero HTTP 429s (one transient 502, auto-retried); OSF does not expose X-RateLimit-* headers. Each request takes ~1.5s. This actor paces requests (short delay, page[size]=50, batches ≤ 50) and, if a 429 ever occurs, fails with a distinct rate-limit message. For very large jobs, add an OSF token upstream if you need higher throughput (not required here).

Related actors

• Figshare Scraper
• Zenodo Scraper
• Dryad Datasets Scraper
• Semantic Scholar Scraper

Local development

$python3 tests/test_mapping.py   # offline smoke tests (stdlib only)

No API key. Data © their respective authors; OSF content is generally openly licensed — check each record's license.