# Academic Research Aggregator Scraper (`parseforge/academic-research-aggregator-scraper`) Actor

Queries Crossref journals, OpenAlex institutions and topics, and Dryad datasets from a single keyword. Returns each paper, dataset, or institution as a flat row with title, authors, DOI, and abstract.

- **URL**: https://apify.com/parseforge/academic-research-aggregator-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Education, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Academic Research Aggregator Scraper

**Aggregate research papers, datasets, and institutions from Crossref, OpenAlex, and Dryad in one run.** Feed in a keyword or topic and get a unified stream of DOIs, titles, authors, and abstracts. No API keys or journal subscriptions required.

Finding papers across siloed academic databases means repeating the same keyword on Crossref, OpenAlex, and Dryad, then merging the results by hand. This Actor queries all three catalogues in parallel from a single research topic or keyword, deduplicates the hits, and returns one clean dataset. It works for literature reviews, meta-analyses, and institutional benchmarking without writing a line of code.

| Who uses it | What they scrape Academic Research Aggregator for |
|---|---|
| PhD students and postdocs | Run a quick literature scan across multiple databases for a dissertation chapter. |
| Research librarians | Build a bibliography on a faculty member's topic from several catalogues at once. |
| Data journalists | Pull the latest datasets and papers on a public-health or climate keyword. |
| University strategy offices | Benchmark institutional research output by country and type from OpenAlex. |

### What it does

This Actor queries Crossref journals, OpenAlex institutions and topics, and Dryad datasets for a given keyword, and returns each matching paper, dataset, or institution as a flat row.

- 📚 **Multi-source aggregation:** Query Crossref, OpenAlex institutions, OpenAlex topics, and Dryad datasets in one run.
- 🏛️ **Institution filtering:** Narrow OpenAlex results to a country code and an institution type such as education, healthcare, or company.
- 🔢 **Flexible volume:** Grab up to 10 papers on the free tier for a preview, or scale to a million items on a paid plan.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Academic Research Aggregator data

**📖 Systematic literature review.**

A PhD candidate enters a dissertation keyword, selects all four sources, and exports a deduplicated CSV of papers and datasets for their PRISMA flowchart.

**🏥 Public-health evidence scan.**

A policy researcher runs a disease or intervention keyword across Crossref and Dryad to collect the latest papers and underlying datasets in one afternoon.

**🏫 Institutional benchmarking.**

A university strategy officer sets the OpenAlex country filter to 'gb' and type to 'education' to compare their institution's topic coverage against UK peers.

**📊 Dataset discovery for replication.**

A postdoc searches Dryad and Crossref for a specific analysis keyword to find both the original paper and the associated open data.

### Why choose this scraper

|  | What you get |
|---|---|
| **One keyword, four catalogues** | Crossref, OpenAlex institutions, OpenAlex topics, and Dryad datasets queried in parallel. |
| **Unified output schema** | Every result lands in the same flat structure, ready for CSV, JSON, Excel, or XML export. |
| **No API keys or subscriptions** | Reads public APIs directly; no journal paywalls or developer registrations needed. |
| **Country and type filters** | Restrict OpenAlex institutions to a 2-letter country code and a specific institution type. |

### How it compares

No other Store actor targets Academic Research Aggregator the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Academic Research Aggregator Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Academic Research Aggregator changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor with a research topic or keyword, pick which academic sources to query, and optionally filter OpenAlex institutions by country and type. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "maxItems": 10,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ]
}
```

A larger pull:

```json
{
  "maxItems": 200,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ]
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Academic Research Aggregator Scraper](https://apify.com/parseforge/academic-research-aggregator-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Academic Research Aggregator through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/academic-research-aggregator-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that at least one source is enabled in the input. Try a broader keyword and remove the OpenAlex country or institution type filter. Some niche terms return few hits across all catalogues.

**Why does the run stop at 10 items?**

Free Apify accounts are capped at 10 items for this Actor. Upgrade to a paid plan and set maxItems to a higher number to collect more results.

**Why are some fields empty in my dataset?**

Different sources provide different metadata. Dryad datasets may not have an abstract, and Crossref records may lack institution types. Empty fields are expected when a source does not supply that attribute.

**Why does my OpenAlex country filter not work?**

Use an ISO 2-letter country code in lowercase, such as 'us' or 'gb'. The filter only applies to the OpenAlex institutions source. It is ignored for Crossref, OpenAlex topics, and Dryad.

**Why do I see duplicate DOIs?**

The Actor queries sources in parallel and does not deduplicate. If the same paper appears in Crossref and OpenAlex, you will see two rows. Deduplicate by DOI in your spreadsheet or analysis tool.

### FAQ

| Question | Answer |
|---|---|
| Which academic databases does this Actor search? | It queries Crossref journals, OpenAlex institutions, OpenAlex topics, and Dryad datasets. You can enable or disable each source in the input. |
| Do I need a university subscription or API key? | No. The Actor reads the public APIs of Crossref, OpenAlex, and Dryad. No login, no API key, and no institutional proxy are required. |
| How many results can I get in one run? | Free Apify users are limited to 10 items for a preview. Paid users can set maxItems up to 1,000,000 and pull a much larger corpus. |
| Can I filter by country or institution type? | Yes. Use the optional OpenAlex country code field with an ISO 2-letter code such as 'us' or 'de', and pick an institution type like 'education' or 'healthcare'. |
| What does the output look like? | Every result is a flat row with fields such as title, authors, DOI, abstract, publication date, and source catalogue. You can export to CSV, JSON, Excel, or XML. |
| Does it deduplicate papers that appear in more than one source? | The Actor queries each source independently and returns the raw hits. Deduplication by DOI or title can be done in a spreadsheet or with a simple post-processing script. |
| Can I search for a specific journal or dataset keyword? | Yes. The query field accepts journal names, topic keywords, or dataset keywords. The Actor passes your term to every source you have enabled. |
| Is this Actor suitable for a meta-analysis? | It is a good first-pass aggregation tool for a meta-analysis. It collects candidate papers and datasets across multiple databases, which you then screen and code offline. |
| What is the difference between OpenAlex institutions and OpenAlex topics? | OpenAlex institutions returns research organisations matching your keyword. OpenAlex topics returns concepts and fields of study. Enable one or both depending on your goal. |
| Does it scrape full-text PDFs? | No. The Actor collects metadata such as titles, abstracts, DOIs, and authors. It does not download or scrape full-text PDFs. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Crossref, OpenAlex, and Dryad. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `query` (type: `string`):

Optional keyword (journal name, topic, dataset keyword).

## `maxItems` (type: `integer`):

Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000

## `sources` (type: `array`):

Academic sources to query in parallel.

## `openalexCountry` (type: `string`):

ISO 2-letter country code to filter institutions (e.g. 'us', 'gb', 'de').

## `openalexInstitutionType` (type: `string`):

OpenAlex institution type filter.

## Actor input object example

```json
{
  "maxItems": 10,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ],
  "openalexInstitutionType": ""
}
```

# Actor output Schema

## `results` (type: `string`):

Aggregated academic research metadata

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "sources": [
        "crossref",
        "openalex_institutions",
        "openalex_topics",
        "datadryad"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/academic-research-aggregator-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "sources": [
        "crossref",
        "openalex_institutions",
        "openalex_topics",
        "datadryad",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/academic-research-aggregator-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ]
}' |
apify call parseforge/academic-research-aggregator-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/academic-research-aggregator-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/UveFEbRDCf3tlMisI/builds/WKhzTjgGXpsl04ULG/openapi.json