Harvard Dataverse Datasets Scraper
Pricing
from $7.50 / 1,000 results
Harvard Dataverse Datasets Scraper
Scrapes Harvard Dataverse datasets, files, and dataverses by search query. Returns title, description, authors, publication date, subjects, identifiers, and links.
Pricing
from $7.50 / 1,000 results
Rating
0.0
(0)
Developer
ParseForge
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
12 days ago
Last modified
Categories
Share
Harvard Dataverse Datasets Scraper
Scrape Harvard Dataverse datasets by keyword, type, or up to a million records per run. Each dataset comes with its title, description, authors, publication date, subjects, identifiers, and links. No login or API key. Export to CSV, JSON, Excel, or XML.
Harvard Dataverse hosts open research data across every discipline, but its web interface makes bulk collection slow. This Actor reads the public search results directly, filtered by keyword and record type, and returns each match in one fixed schema.
| Who uses it | What they scrape Harvard Dataverse for |
|---|---|
| Academic researchers | Build a corpus of datasets for a literature review or meta-analysis |
| Data journalists | Find open datasets on a topic to support an investigation |
| Librarians | Curate lists of datasets for subject guides |
| Data scientists | Collect training or benchmark datasets for model development |
What it does
This Actor collects Harvard Dataverse datasets, files, and dataverses by search query and returns each one as a flat row.
- ๐ Keyword search: free-text query matched against dataset titles and descriptions.
- ๐๏ธ Record type filter: restrict results to datasets, files, or dataverses.
- ๐ฆ Bulk collection: set a maximum up to 1,000,000 records per run.
- ๐ Flat output: every record is one row, ready for CSV, JSON, Excel, or XML.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with Harvard Dataverse data
๐ Build a literature review corpus.
A PhD student searches for 'climate migration' and collects 500 dataset records with titles, authors, and subjects to map what data exists.
๐ฐ Find data for a news story.
A journalist queries 'opioid prescriptions' and exports a CSV of datasets to identify sources for an investigation.
๐งช Gather benchmark datasets.
A machine learning engineer filters by type 'dataset' and keyword 'satellite imagery' to assemble a training set.
๐๏ธ Curate a subject guide.
A librarian runs a query for 'public health' and uses the results to populate a research guide with links and descriptions.
Why choose this scraper
| What you get | |
|---|---|
| No API registration | Uses the public search endpoint, no token or OAuth setup |
| Fixed schema | Every record has the same fields, so joins and pivots are trivial |
| Bulk export | One run can return up to a million records in your chosen format |
How it compares
This Actor focuses on bulk collection with a simple keyword and type filter, while the competitors below offer similar search capabilities with different output details.
| Feature | ParseForge | Harvard Dataverse Scraper - Research Datasets | Harvard Dataverse Scraper |
|---|---|---|---|
| Search by keyword | Yes | Yes | Yes |
| Filter by record type | Yes | Not listed | Yes |
| Export names and descriptions | Yes | Yes | Not listed |
| Export authors and publication dates | Yes | Yes | Not listed |
| Export subjects and identifiers | Yes | Yes | Not listed |
| Bulk collection up to 1,000,000 records | Yes | Not listed | Not listed |
Configure the run
Drive the Actor with a search query and an optional record type filter, and set a maximum number of records to collect per run. The Input tab lists every parameter.
A first run with the defaults:
A larger pull:
Pricing
Pay-per-result: $0.0085 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $0.85 |
| 1,000 results | $8.50 |
| 10,000 results | $85.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the Harvard Dataverse Datasets Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to Harvard Dataverse through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check that your query is not too restrictive. Try a broader keyword or remove the type filter. Also verify that the query is spelled correctly.
Why did the run stop before reaching maxItems?
The Actor stops when there are no more matching records. If you expected more, broaden your query or remove the type filter.
Can I get more than 1,000,000 records?
The maximum is 1,000,000 per run. If you need more, split your query into smaller chunks and run multiple times.
Why are some fields empty in the output?
Not all records have every field populated. Empty fields mean the metadata was not provided by the repository.
The run failed with a timeout. What should I do?
Reduce maxItems or narrow your query to reduce the load. If the problem persists, contact support.
FAQ
| Question | Answer |
|---|---|
| Do I need a Harvard Dataverse API key? | No. This Actor uses the public search interface, so no registration or token is required. |
| Can I filter by author or subject? | The input schema supports a free-text query and a record type filter. Author and subject are returned in the output, but not used as input filters. |
| What is the maximum number of records I can get? | You can set maxItems up to 1,000,000 per run. The actual number returned depends on how many records match your query. |
| What formats can I export to? | The Actor output can be exported to CSV, JSON, Excel, or XML from the Apify platform. |
| Does it scrape files or only dataset metadata? | It scrapes metadata for datasets, files, and dataverses. It does not download the actual data files. |
| Can I search in languages other than English? | The query is passed to Harvard Dataverse as-is, so it will match metadata in any language present in the repository. |
| Is this an official Harvard Dataverse API? | No, this is a third-party scraper built on the public search endpoint. It is not affiliated with Harvard University. |
| How often is the data updated? | Each run fetches live data from Harvard Dataverse at that moment. Schedule the Actor to run periodically for fresh results. |
| Can I get the full text of dataset descriptions? | Yes, the description field is included in the output for each record. |
| What is the difference between dataset, file, and dataverse types? | A dataset is a collection of files with metadata. A file is a single data file. A dataverse is a container that holds datasets and other dataverses. |
Related actors
Browse the full ParseForge collection for more scrapers.
๐ Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
โ ๏ธ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Harvard University. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
