Zenodo Scraper: Datasets, Software & Research Output

Zenodo Records Scraper searches Zenodo, CERN's open research repository, and returns one structured record per deposit, DOI, title, plain-text description, creators with ORCIDs, licence, keywords, communities, file listings with direct download URLs, version, related identifiers and the public download and view counts.

Zenodo runs a documented REST API over its entire catalogue of 7 million records. This Actor reads it directly: no browser, no proxies, no login, no API token.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/zenodo-records-scraper.md

What you get

Output field	Meaning
recordId, doi	Zenodo record ID and the minted DOI
title	Record title
description	Abstract as clean plain text, HTML stripped, entities decoded
creators	Authors with name, affiliation and orcid
resourceType	dataset, software, publication, poster, image, …
publicationDate	Publication date as YYYY-MM-DD
license	SPDX-style licence id (cc-by-4.0, mit, …)
keywords	Depositor-supplied keywords
communities	Community slugs the record belongs to
files	File name, byte size, checksum and direct download URL
downloads, views	Public usage statistics, unusual for a repository to expose
version	Version string, when the depositor set one
relatedIdentifiers	Linked DOIs/URLs with their relation type (isSupplementTo, isPartOf, …)
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the filters used, and any query that failed.

Common use cases

• Find open datasets for ML training. Filter to dataset, sort through by download count, and pull the file URLs straight from the output.
• Track research software releases. Filter to software and watch a community or keyword on a schedule.
• Monitor a lab's data publications. Search the lab name or watch its Zenodo community.
• RAG and knowledge-base ingestion. Clean plain-text abstracts with DOIs make good chunks and good citations.
• Bibliometrics. Download and view counts per record are exposed publicly, so impact analysis needs no extra source.

Quick start

Open datasets for a topic:

{
 "searchQueries": ["machine learning"],
 "resourceTypes": ["dataset"],
 "maxResults": 500
}

Everything a community has published, with file listings:

{
 "communities": ["biosyslit"],
 "includeFiles": true,
 "maxResults": 1000
}

Two topics across two resource types, four independent searches:

{
 "searchQueries": ["\"climate model\"", "genome assembly"],
 "resourceTypes": ["dataset", "software"],
 "includeFiles": false,
 "maxResults": 2000
}

Input

Field	Type	Default	Notes
searchQueries	array	[]	Full-text search terms. Quote a phrase to match it exactly. Empty = no text filter.
resourceTypes	array	[]	dataset, software, publication, poster, presentation, image, video, lesson, physicalobject, other. Empty = all types.
communities	array	[]	Community slugs or full zenodo.org/communities/<slug> URLs. Empty = all of Zenodo.
includeFiles	boolean	true	Include file listings with sizes, checksums and download URLs.
maxResults	integer	500	Total cap across all query combinations, split evenly. 0 = no limit.

How the filters combine. Zenodo applies only the last value when a facet is repeated, so this Actor runs the cross product as separate searches: two queries × two types = four independent searches, each independently paginated and independently retried. That is why maxResults is split between combinations, and why one bad query cannot take down the others.

Output example

{
 "recordId": 21805794,
 "doi": "10.5281/zenodo.21805794",
 "title": "Submerged Aquatic Vegetation (SAV) in the Balearic Islands",
 "description": "Dataset description\n\nAnnual map of submerged aquatic vegetation (SAV) along the coast of the Balearic Islands…",
 "creators": [
 { "name": "König, Marcel", "affiliation": "Brockmann Consult (Germany)", "orcid": "0000-0002-7617-888X" }
 ],
 "resourceType": "dataset",
 "publicationDate": "2026-08-05",
 "license": "cc-by-4.0",
 "keywords": ["remote sensing", "Sentinel-2"],
 "communities": ["eu", "foccus-project"],
 "files": [
 {
 "name": "UMPSupportingDoc_BAT_V1.pdf",
 "size": 947143,
 "checksum": "md5:9856d723eea345ec504d88a32d16d7f1",
 "downloadUrl": "https://zenodo.org/api/records/21128664/files/UMPSupportingDoc_BAT_V1.pdf/content"
 }
 ],
 "downloads": 0,
 "views": 105,
 "version": "1.0",
 "relatedIdentifiers": [
 { "identifier": "10.5281/zenodo.17579480", "relation": "isDescribedBy", "scheme": "doi" }
 ],
 "scrapedAt": "2026-08-06T11:47:54.497Z"
}

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~zenodo-records-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "searchQueries": ["protein structure"],
 "resourceTypes": ["dataset"],
 "maxResults": 200
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/zenodo-records-scraper').call({
 searchQueries: ['machine learning'],
 resourceTypes: ['dataset', 'software'],
 maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const rec of items) console.log(`${rec.doi}, ${rec.title} (${rec.downloads} downloads)`);

Limits and behaviour

• 25 records per page, and that is Zenodo's rule, not ours. Anonymous callers are capped at 25; the documented 100 needs an API token. This Actor stays token-free and pages at 25.
• 10 000-record result window. Zenodo rejects any request where page × size exceeds 10 000. The Actor stops one page short and logs a warning telling you to narrow the query rather than letting the API 400.
• Zenodo is slow. Filtered searches routinely take 20-30 s and answer 504 under load. Requests allow 90 s, and 429/5xx/504 are retried with exponential backoff, a 429 honours the Retry-After header rather than guessing.
• Rate limit is ~30 requests/minute for anonymous callers, so a one-second pause sits between pages.
• Failures never abort the run. A query that fails is recorded in RUN_SUMMARY.failures and the remaining combinations continue. The Actor only throws if every combination fails.
• Descriptions are decoded twice. Zenodo stores depositor-authored HTML, so entities are decoded before tags are stripped and again afterwards. The full Latin-1 entity block is handled, which matters for accented author names (König → König) and units (° → °).
• Public data only. No authentication, no personal data, no access-control bypass.

FAQ

Do I need a proxy? No. Proxy configuration is not required to run this Actor.

Do I need a Zenodo account or API token? No. You supply no credentials.

Why is files empty on some records? Those deposits are closed or restricted access. The metadata is public; the files are not.

Why did I get fewer records than maxResults? The budget is split evenly across query combinations, and a combination stops early when its result set is exhausted. RUN_SUMMARY reports the per-run totals.

Can I search by author or DOI? Yes, searchQueries is passed straight to Zenodo's search, so creators.name:"Smith" and doi:"10.5281/zenodo.16537543" both work.

What happens if Zenodo is unavailable? The query is retried with backoff, then recorded in RUN_SUMMARY.failures; the run continues with the remaining combinations.

Can I schedule it? Yes, it is designed for scheduled runs. Sort order is mostrecent, so a scheduled run sees new deposits first.

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.