# Zenodo Scraper: Datasets, Software & Research Output (`arman-bd/zenodo-records-scraper`) Actor

Scrape Zenodo records: datasets, software, publications and posters with DOI, licence, files, version history and public download stats. No login, no proxy, no browser.

- **URL**: https://apify.com/arman-bd/zenodo-records-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Developer tools, AI, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.56 / 1,000 record scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Zenodo Scraper: Datasets, Software & Research Output

![Zenodo Scraper: One record per Zenodo deposit, DOI, licence, creators, files and the public download and view counts](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/zenodo-records-scraper.jpg)

**Zenodo Records Scraper** searches **Zenodo**, CERN's open research repository, and returns one structured record per deposit, DOI, title, plain-text description, creators with ORCIDs, licence, keywords, communities, file listings with direct download URLs, version, related identifiers and the **public download and view counts**.

Zenodo runs a documented REST API over its entire catalogue of 7 million records. This Actor reads it directly: **no browser, no proxies, no login, no API token.**

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/zenodo-records-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/zenodo-records-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `recordId`, `doi` | Zenodo record ID and the minted DOI |
| `title` | Record title |
| `description` | Abstract as clean plain text, HTML stripped, entities decoded |
| `creators` | Authors with `name`, `affiliation` and `orcid` |
| `resourceType` | `dataset`, `software`, `publication`, `poster`, `image`, … |
| `publicationDate` | Publication date as `YYYY-MM-DD` |
| `license` | SPDX-style licence id (`cc-by-4.0`, `mit`, …) |
| `keywords` | Depositor-supplied keywords |
| `communities` | Community slugs the record belongs to |
| `files` | File name, byte size, checksum and direct download URL |
| `downloads`, `views` | **Public usage statistics**, unusual for a repository to expose |
| `version` | Version string, when the depositor set one |
| `relatedIdentifiers` | Linked DOIs/URLs with their relation type (`isSupplementTo`, `isPartOf`, …) |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, and any query that failed.

### Common use cases

- **Find open datasets for ML training.** Filter to `dataset`, sort through by download count, and pull the file URLs straight from the output.
- **Track research software releases.** Filter to `software` and watch a community or keyword on a schedule.
- **Monitor a lab's data publications.** Search the lab name or watch its Zenodo community.
- **RAG and knowledge-base ingestion.** Clean plain-text abstracts with DOIs make good chunks and good citations.
- **Bibliometrics.** Download and view counts per record are exposed publicly, so impact analysis needs no extra source.

### Quick start

Open datasets for a topic:

```json
{
 "searchQueries": ["machine learning"],
 "resourceTypes": ["dataset"],
 "maxResults": 500
}
```

Everything a community has published, with file listings:

```json
{
 "communities": ["biosyslit"],
 "includeFiles": true,
 "maxResults": 1000
}
```

Two topics across two resource types, four independent searches:

```json
{
 "searchQueries": ["\"climate model\"", "genome assembly"],
 "resourceTypes": ["dataset", "software"],
 "includeFiles": false,
 "maxResults": 2000
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `searchQueries` | array | `[]` | Full-text search terms. Quote a phrase to match it exactly. Empty = no text filter. |
| `resourceTypes` | array | `[]` | `dataset`, `software`, `publication`, `poster`, `presentation`, `image`, `video`, `lesson`, `physicalobject`, `other`. Empty = all types. |
| `communities` | array | `[]` | Community slugs or full `zenodo.org/communities/<slug>` URLs. Empty = all of Zenodo. |
| `includeFiles` | boolean | `true` | Include file listings with sizes, checksums and download URLs. |
| `maxResults` | integer | `500` | Total cap across all query combinations, split evenly. `0` = no limit. |

**How the filters combine.** Zenodo applies only the last value when a facet is repeated, so this Actor runs the **cross product** as separate searches: two queries × two types = four independent searches, each independently paginated and independently retried. That is why `maxResults` is split *between* combinations, and why one bad query cannot take down the others.

### Output example

```json
{
 "recordId": 21805794,
 "doi": "10.5281/zenodo.21805794",
 "title": "Submerged Aquatic Vegetation (SAV) in the Balearic Islands",
 "description": "Dataset description\n\nAnnual map of submerged aquatic vegetation (SAV) along the coast of the Balearic Islands…",
 "creators": [
 { "name": "König, Marcel", "affiliation": "Brockmann Consult (Germany)", "orcid": "0000-0002-7617-888X" }
 ],
 "resourceType": "dataset",
 "publicationDate": "2026-08-05",
 "license": "cc-by-4.0",
 "keywords": ["remote sensing", "Sentinel-2"],
 "communities": ["eu", "foccus-project"],
 "files": [
 {
 "name": "UMPSupportingDoc_BAT_V1.pdf",
 "size": 947143,
 "checksum": "md5:9856d723eea345ec504d88a32d16d7f1",
 "downloadUrl": "https://zenodo.org/api/records/21128664/files/UMPSupportingDoc_BAT_V1.pdf/content"
 }
 ],
 "downloads": 0,
 "views": 105,
 "version": "1.0",
 "relatedIdentifiers": [
 { "identifier": "10.5281/zenodo.17579480", "relation": "isDescribedBy", "scheme": "doi" }
 ],
 "scrapedAt": "2026-08-06T11:47:54.497Z"
}
```

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~zenodo-records-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "searchQueries": ["protein structure"],
 "resourceTypes": ["dataset"],
 "maxResults": 200
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/zenodo-records-scraper').call({
 searchQueries: ['machine learning'],
 resourceTypes: ['dataset', 'software'],
 maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const rec of items) console.log(`${rec.doi}, ${rec.title} (${rec.downloads} downloads)`);
```

### Limits and behaviour

- **25 records per page, and that is Zenodo's rule, not ours.** Anonymous callers are capped at 25; the documented 100 needs an API token. This Actor stays token-free and pages at 25.
- **10 000-record result window.** Zenodo rejects any request where `page × size` exceeds 10 000. The Actor stops one page short and logs a warning telling you to narrow the query rather than letting the API 400.
- **Zenodo is slow.** Filtered searches routinely take 20-30 s and answer 504 under load. Requests allow 90 s, and 429/5xx/504 are retried with exponential backoff, a 429 honours the `Retry-After` header rather than guessing.
- **Rate limit is ~30 requests/minute** for anonymous callers, so a one-second pause sits between pages.
- **Failures never abort the run.** A query that fails is recorded in `RUN_SUMMARY.failures` and the remaining combinations continue. The Actor only throws if *every* combination fails.
- **Descriptions are decoded twice.** Zenodo stores depositor-authored HTML, so entities are decoded before tags are stripped and again afterwards. The full Latin-1 entity block is handled, which matters for accented author names (`K&ouml;nig` → `König`) and units (`&deg;` → `°`).
- **Public data only.** No authentication, no personal data, no access-control bypass.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need a Zenodo account or API token?** No. You supply no credentials.

**Why is `files` empty on some records?** Those deposits are closed or restricted access. The metadata is public; the files are not.

**Why did I get fewer records than `maxResults`?** The budget is split evenly across query combinations, and a combination stops early when its result set is exhausted. `RUN_SUMMARY` reports the per-run totals.

**Can I search by author or DOI?** Yes, `searchQueries` is passed straight to Zenodo's search, so `creators.name:"Smith"` and `doi:"10.5281/zenodo.16537543"` both work.

**What happens if Zenodo is unavailable?** The query is retried with backoff, then recorded in `RUN_SUMMARY.failures`; the run continues with the remaining combinations.

**Can I schedule it?** Yes, it is designed for scheduled runs. Sort order is `mostrecent`, so a scheduled run sees new deposits first.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `searchQueries` (type: `array`):

Full-text search terms. Each query runs as its own search. two queries means two independent result sets, not one AND'd query. Quote a phrase to match it exactly. Leave empty to browse without a text filter.

## `resourceTypes` (type: `array`):

Keep only records of these Zenodo resource types. Each type runs as its own search. Leave empty for every type.

## `communities` (type: `array`):

Zenodo community identifiers. the slug in https://zenodo.org/communities/<slug>. Full community URLs are accepted and normalized. Each community runs as its own search. Leave empty for all of Zenodo.

## `includeFiles` (type: `boolean`):

Include each record's file list with names, byte sizes, checksums and direct download URLs. Disable for a smaller, faster dataset.

## `maxResults` (type: `integer`):

Cap the total number of records saved across all query combinations. The budget is split evenly between them. Set 0 for no limit.

## Actor input object example

```json
{
  "searchQueries": [
    "\"climate model\"",
    "genome assembly"
  ],
  "resourceTypes": [
    "dataset",
    "software"
  ],
  "communities": [
    "biosyslit",
    "eu"
  ],
  "includeFiles": true,
  "maxResults": 500
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "machine learning"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/zenodo-records-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQueries": ["machine learning"] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/zenodo-records-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "machine learning"
  ]
}' |
apify call arman-bd/zenodo-records-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/zenodo-records-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CkPbBKn7zK6WVB276/builds/qH3nAIePUbe56tu9d/openapi.json