OSF Open Science Framework Scraper

OSF Open Science Framework Scraper

Scrape public research projects, preprints, and registrations from the Open Science Framework, up to a million per run. Each record comes with its title, authors, DOI, subjects, and full metadata. No API key required. Export to CSV, JSON, Excel, or XML.

The OSF Open Science Framework hosts millions of open-access research outputs, but its API requires pagination handling and rate-limit management. This scraper reads the public feeds directly, filtered by resource type, keyword, preprint provider, or subject, and returns each match in one fixed schema.

Who uses it	What they scrape OSF for
Meta-researchers	Finding replication studies and pre-registered designs across disciplines.
Librarians	Building institutional preprint collections by provider and subject.
Data journalists	Tracking the growth of open science practices over time.
Academic administrators	Monitoring faculty research outputs deposited in public repositories.

What it does

This Actor collects OSF research projects, preprints, or registrations by keyword, provider, or subject, and returns each one as a flat row.

• 📋 Three resource modes: collect research projects (nodes), preprints, or registrations with one selector.
• 🔍 Keyword search: filter by title, description, or tags with a free-text query.
• 🏷️ Subject filtering: narrow preprints and registrations by comma-separated subject tags like 'Psychology,Education'.
• 📦 Provider targeting: restrict preprints to a specific community provider such as psyarxiv, socarxiv, or engrxiv.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with OSF data

📈 Monitor open science trends.

A meta-researcher runs the scraper weekly with the keyword 'replication' in registrations mode to count new pre-registered replication attempts.

📚 Build a subject repository.

A librarian collects all PsyArXiv preprints tagged 'Clinical Psychology' to populate an institutional preprint portal.

🔎 Discover pre-registered studies.

A graduate student searches registrations by keyword to find existing pre-registrations before designing their own study.

📊 Audit institutional output.

A research administrator searches projects by their university name to track public deposits by faculty.

Why choose this scraper

	What you get
No API key	Reads public OSF feeds directly, no registration or token needed.
Fixed schema	Every record returns the same flat structure for easy analysis.
Bulk collection	Set a high maxItems value and collect up to a million records per run.
Multi-format export	Download your dataset as CSV, JSON, Excel, or XML.

How it compares

No other Store actor targets OSF the same way, so the honest comparison is with the alternatives teams actually weigh.

	OSF Open Science Framework Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When OSF changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor with a keyword query and choose a resource mode, then optionally narrow results by preprint provider or subject tags. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10,
  "mode": "preprints",
  "query": "replication study"
}

A larger pull:

{
  "maxItems": 200,
  "mode": "preprints",
  "query": "replication study"
}

Pricing

Pay-per-result: $0.01499 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.50
1,000 results	$14.99
10,000 results	$149.90

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the OSF Open Science Framework Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to OSF through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/osf-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your keyword and filters match existing OSF content. Try a broader query or remove the provider and subject filters to see if results appear.

The provider filter is not working.

Ensure you are in preprints mode. The provider filter only applies to preprints. Also verify the provider slug is spelled correctly, e.g. 'psyarxiv' not 'PsyArXiv'.

I set maxItems to 1000 but only got 200 records.

The scraper stops when it has collected the requested number or when no more matching records exist. Your query likely returned only 200 matches.

The run is taking a long time.

Large maxItems values and broad queries can take time. Reduce maxItems or add filters to narrow the search and speed up collection.

FAQ

Question	Answer
Do I need an OSF account or API key?	No. The scraper reads the public OSF feeds, so no login, account, or API key is required.
What is the difference between nodes, preprints, and registrations?	Nodes are general research projects. Preprints are open-access papers shared before peer review. Registrations are time-stamped, pre-registered study designs.
Can I filter by a specific preprint provider?	Yes. In preprints mode, enter a provider slug like 'psyarxiv', 'socarxiv', or 'engrxiv' to restrict results to that community.
How do subject filters work?	Enter comma-separated subjects such as 'Psychology,Education'. The filter applies to preprints and registrations and matches against the OSF subject taxonomy.
How many records can I collect in one run?	You can set maxItems up to 1,000,000. The actual number collected depends on how many results match your query and filters.
What output formats are supported?	The dataset can be exported to CSV, JSON, Excel, or XML from the Apify platform.
Does this scraper get full-text PDFs?	No. It collects metadata such as title, authors, DOI, and subjects. Full-text files are not downloaded.
Can I search without a keyword?	Yes. Leave the keyword field empty to browse all records in the selected mode, optionally filtered by provider or subject.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Center for Open Science. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.