Research Repository Harvester — Dataverse / DSpace / InvenioRDM

Export dataset and publication metadata from a public research repository into one flat table: DOI or handle, title, authors, resource type, licence, file count and landing page. It works with sites running Dataverse, DSpace 7/8 or InvenioRDM / Zenodo, so the same columns come back whether you search Zenodo, Harvard Dataverse or a university DSpace.

Typical jobs: build a reading list or data-reuse shortlist for a topic, check which licences a set of deposits carry, feed DOIs and landing pages into a citation or cataloguing tool, or give an AI agent a clean list of candidate datasets to inspect.

It returns metadata only. It does not download data files, and it searches one repository site per run.

Quick start: 25 Zenodo climate records

1. Open the ready-made example Export Zenodo climate dataset metadata and click Try for free, or paste this into the Input tab (it is also the Actor's default example):

{ "platform": "invenio", "baseUrl": "https://zenodo.org", "query": "climate", "maxRecords": 25, "pageSize": 25 }

2. Start the run. On 2026-09-24 this exact input returned 25 records in about 6 seconds (owner cloud run on build 0.1.4, 25 record events).
3. Open the Output tab and export CSV, JSON or Excel.
4. Change query, or switch platform + baseUrl to another site (see the examples below).

What the output looks like

Three rows from that run, trimmed (the raw column holds the full original record and is omitted here):

persistentId	title	authors	type	license	fileCount	publicationDate
10.5281/zenodo.16537543	SYMBA D6.1 - Report on system mapping and SYMBA forum activities	CLIMATE KIC	Project deliverable	cc-by-4.0	1	2025-07-28
10.5281/zenodo.6050259	Murrayon hastatus	Kaczmarek, Łukasz	Taxonomic treatment	cc-zero	1	2017-06-28
10.5281/zenodo.16537726	SYMBA D6.3 - 1st Report on summary of New European Bauhaus activities	CLIMATE KIC	Project deliverable	cc-by-4.0	1	2025-07-28

Every row also has landingPage (for example https://zenodo.org/records/16537543), repoId, platform, baseUrl, description and subjects.

What to expect from that same 25-row run: types were Dataset 9, Software 6, Project deliverable 4, plus a few others; subjects was empty on 17 rows, license on 2 and description on 3. Zenodo's keyword search matches broadly, so results mix datasets, software and reports. Filter on type afterwards, or use a more specific query. description is HTML as supplied by the repository.

Pricing

Pay per event: one record event per row delivered, no start fee. The listed price is $0.0005 per record on the free plan ($0.50 per 1,000), falling to $0.00025 on the highest Apify plan. The 25-record quick start is about $0.0125 in record fees at the free-plan price. Check the Pricing tab for the current figure. If you set a maximum charge for the run, the Actor stops fetching pages once no further record fits.

Input

Field	Required	Description
platform	✅	dataverse, dspace, or invenio
baseUrl	✅	Site root, e.g. https://dataverse.harvard.edu, https://dspace.mit.edu, https://zenodo.org
query		Free-text search (empty = browse the site's default listing)
maxRecords		Maximum records to return (default: one page)
pageSize		Records per API page, 1–100 (default 25)
bearerToken		Optional token for a gated install (never needed for public search, never logged)
extraHeaders		Optional extra HTTP headers as a JSON object

Other sites

Harvard Dataverse (an owner cloud run of this with maxRecords: 7 returned 7 records):

{ "platform": "dataverse", "baseUrl": "https://dataverse.harvard.edu", "query": "climate", "maxRecords": 100 }

A DSpace site: point at the site root, and the Actor appends /server/api. Only DSpace 7 or later is supported:

{ "platform": "dspace", "baseUrl": "https://dspace.mit.edu", "query": "climate", "maxRecords": 100 }

Limits

• One site per run; there is no cross-repository search or de-duplication.
• Coverage and ordering are whatever the site's own search API returns. This is not a full-history or complete-catalogue guarantee.
• Fields a site or record does not expose come back as null.
• Older DSpace (6.x and earlier), custom repository software, and sites behind bot protection are not supported. A wrong or non-repository baseUrl fails fast with an error instead of an empty "success".
• Respect each repository's terms and each record's licence when you reuse the metadata.

How it works

For each platform the actor calls its uniform search endpoint, pages through the result set, and maps each record into the unified schema:

• Dataverse → GET {base}/api/search?q=…&type=dataset&per_page=N&start=M (paginated by record offset), with a {base}/api/info/version handshake.
• DSpace → GET {base}/server/api/discover/search/objects?query=…&page=N&size=M (HAL/JSON envelope; Dublin-Core dc.* metadata flattened), with a {base}/server/api handshake that reports dspaceName / dspaceVersion.
• Invenio → GET {base}/api/records?q=…&size=M&page=N; handles both the legacy Zenodo record shape and the modern InvenioRDM shape transparently.

It returns catalog metadata only (what datasets exist and where to get them), never the underlying data files. A valid install with no matches returns 0 records and exits cleanly; a non-repository host (an HTML anti-bot page, a wrong URL, or a status:ERROR envelope) fails fast with a clear error rather than a silent empty run. No authentication is required for public repository search.