# Scholarly Papers Scraper: OpenAlex and arXiv (`parselab/openalex-arxiv-scholarly-works-scraper`) Actor

Scrape scholarly papers from OpenAlex and arXiv. Get abstracts, citations, field weighted impact, open access links, topics, authors with institutions and countries, funders and retraction flags, plus author and institution profiles with h-index. Export to CSV or Excel.

- **URL**: https://apify.com/parselab/openalex-arxiv-scholarly-works-scraper.md
- **Developed by:** [ParseLab](https://apify.com/parselab) (community)
- **Categories:** Other, News
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $13.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Scholarly Papers Scraper: OpenAlex Works, Authors, Institutions and arXiv Preprints

This scholarly scraper brings two open sources of research literature into one tool. OpenAlex is a catalog of more than 250 million papers from every field, with citations, topics, open access status, authors and institutions. arXiv is the preprint server of physics, mathematics, computer science and related fields, where new work appears first. Use one or both to build literature lists, track a field and rank researchers and universities.

For papers you get the abstract, the number of citations, the field weighted citation impact, the percentile among papers of the same field and year, the open access link, the journal, the topic and field, the authors with their institutions and countries, the funders and the retraction flag. For researchers and institutions you get the number of works, citations, h-index and the topics they work on. For preprints you get the abstract, the categories, the dates, the number of versions, the journal reference when a paper was published and the link to the PDF.

Researchers, librarians, R\&D teams, research offices, funders, journalists, investors and AI builders use it to find and rank evidence and people.

### What can you do with this scholarly scraper?

- Find the most cited papers on a topic in the last few years
- Keep only open access papers and get their free links
- Filter by field, country, institution, author, type of work or year
- Rank papers by field weighted citation impact, not only raw citations
- Find international collaborations by the countries of the authors
- List the top researchers of a topic or an institution with their h-index
- Rank universities and companies by works and citations in a country
- Follow new arXiv preprints in a category with their abstracts
- See which preprints were later published in a journal
- Find retracted works
- Export to CSV or Excel, or send new papers to your own tools

### What data can you extract?

| Mode | One row is | Highlights |
|---|---|---|
| arXiv preprints | One preprint | arXiv id, version, title, abstract, authors with affiliations, first and last author, primary category and all categories, submission and update dates, number of versions, author comment, pages and figures mentioned, journal reference, DOI, published flag, PDF and abstract links |
| OpenAlex works | One work | OpenAlex id, DOI, title, abstract, year, date, type, language, journal, source type, publisher, volume, issue, pages, citations, field weighted citation impact, citation percentile, top 1 and top 10 percent flags, references, citations by year, open access flag, status, link, PDF, license and version, authors with ORCID, institutions and countries, number of authors, institutions and countries, international collaboration flag, topic, subfield, field, domain, keywords, sustainable development goals, funders, retraction flag, publication fee |
| OpenAlex authors | One researcher | Name, ORCID, other names, works, citations, h-index, i10 index, recent mean citedness, citations per work, last known institutions, top topics, activity by year |
| OpenAlex institutions | One institution | Name, ROR, country, city, type, website, works, citations, h-index, i10 index, citations per work, top topics, activity by year |

### How to scrape scholarly data

1. Choose **arXiv preprints**, **OpenAlex works**, **OpenAlex authors** or **OpenAlex institutions**.
2. Type search terms, one per line.
3. Add filters: categories for arXiv, or fields, countries, institutions, years, open access and minimum citations for OpenAlex.
4. Paste a free OpenAlex access code for the OpenAlex modes.
5. Set **Max Items** and click **Start**.
6. Export the dataset as CSV, Excel or from the Apify console.

### Input example

```json
{
  "mode": "works",
  "searchTerms": ["retrieval augmented generation"],
  "yearFrom": 2024,
  "fields": ["Computer Science"],
  "openAccessOnly": true,
  "minCitations": 50,
  "sortBy": "citations",
  "maxItems": 100
}
```

### Work output example

```json
{
  "openAlexId": "W4391136507",
  "doi": "10.1145/3641289",
  "title": "A Survey on Evaluation of Large Language Models",
  "publicationYear": 2024,
  "journalOrSource": "ACM Transactions on Intelligent Systems and Technology",
  "citedByCount": 3154,
  "fieldWeightedCitationImpact": 456.58,
  "citationPercentile": 100,
  "isOpenAccess": true,
  "openAccessStatus": "green",
  "field": "Computer Science",
  "authorCount": 16,
  "countries": ["CN", "US", "HK"],
  "isInternationalCollaboration": true
}
```

### Preprint output example

```json
{
  "arxivId": "2610.40454",
  "version": "v1",
  "title": "Glance: The Generator Is Its Own Reranker",
  "authorCount": 2,
  "primaryCategory": "cs.CL",
  "categories": ["cs.CL", "cs.IR"],
  "submittedDate": "2026-10-29",
  "pagesInComment": 12,
  "isPublishedInJournal": false
}
```

### How much does it cost to scrape scholarly data?

You pay per result: $18 per 1,000 rows plus a tiny start fee. The free plan returns up to 10 rows per run. Author and institution rows summarize thousands of works each.

### Tips for better results

- arXiv needs no setup. OpenAlex now gives a very small daily budget to requests without an access code, shared by everyone on the same network address, so paste a free code from openalex.org for the works, authors and institutions modes.
- Field weighted citation impact compares a paper with the average of its field, year and type. A value of 1 is the average, so it is fairer than raw citations across fields.
- Searching everything includes the full text when OpenAlex has it. Switch to title and abstract for tighter results.
- Institution and author names are matched to the first result of OpenAlex. The log shows which one was used. Use the OpenAlex id or the ORCID number to be exact.
- Citations grow over time and OpenAlex updates them regularly, so numbers differ a little from other sources.
- arXiv asks clients to wait three seconds between requests, so a run of 1,000 preprints takes a few minutes.
- A preprint counts as published when its record has a journal reference or a DOI.
- Open access status values are gold, green, hybrid, bronze, diamond and closed.

### Who uses this data?

- **Research offices and funders** measure output, impact and collaboration.
- **R\&D and technology scouting teams** follow new work in a field.
- **Librarians and reviewers** build lists of papers.
- **Investors and journalists** find experts and rising institutions.
- **AI builders** feed abstracts to search and language models.

### Automate and connect

Schedule the Actor every day for an arXiv category and send new preprints to Slack, Google Sheets, Zapier, Make or n8n, or trigger a webhook when a run ends.

### FAQ

**Do I need an account or a login?**
For arXiv, no. For OpenAlex a free access code makes runs reliable. You can get it in a minute at openalex.org.

**Does it return full papers?**
No. It returns abstracts and links. Open access links lead to the free versions.

**Can I search by author?**
Yes. Use the authors filter in the works mode, or the authors mode to see the profile of a researcher.

**What is a field weighted citation impact?**
The citations of a paper divided by the average citations of papers of the same field, year and type.

**How many results can I get?**
As many as match. OpenAlex is read page by page and arXiv in blocks of 100.

**Is this legal advice?**
No. It is public information about published research.

### Legal note

This Actor collects information that is publicly visible on the website. Abstracts remain the property of their authors and publishers. OpenAlex data is released under a public domain license and arXiv asks users to credit the service. Check the source site's terms and your local rules before using the data.

### Support

Missing a field or found a bug? Open an issue from the Actor page.

# Actor input Schema

## `maxItems` (type: `integer`):

Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000.

## `mode` (type: `string`):

arXiv preprints needs no setup. Works, authors and institutions come from OpenAlex, a catalog of more than 250 million papers, and work best with a free OpenAlex access code.

## `searchTerms` (type: `array`):

Words or phrases, for example retrieval augmented generation or perovskite solar cells. Each term is searched on its own. For authors and institutions it is a name.

## `searchIn` (type: `string`):

Works mode. Everything includes the full text when it is available. Title and abstract gives tighter results.

## `arxivCategories` (type: `array`):

arXiv mode. For example cs.AI, cs.LG, stat.ML, physics.optics, q-bio.NC.

## `arxivSort` (type: `string`):

arXiv mode.

## `yearFrom` (type: `integer`):

First publication year.

## `yearTo` (type: `integer`):

Last publication year.

## `dateFrom` (type: `string`):

First publication date, in the form 2025-01-31. It replaces the year.

## `dateTo` (type: `string`):

Last publication date.

## `workTypes` (type: `array`):

Works mode. For example article, review, preprint, book-chapter, dissertation, dataset.

## `fields` (type: `array`):

Works mode. Names such as Medicine, Computer Science, Physics and Astronomy, Economics, Econometrics and Finance.

## `countries` (type: `array`):

Two letter country codes of the authors institutions, for example US, GB, DE, CN. For institutions mode it is the country of the institution.

## `institutions` (type: `array`):

Works and authors modes. Names such as Stanford University or OpenAlex ids such as I97018004. The first match of each name is used.

## `authors` (type: `array`):

Works and arXiv modes. Author names, ORCID numbers or OpenAlex ids.

## `openAccessOnly` (type: `boolean`):

Works mode.

## `openAccessStatuses` (type: `array`):

Works mode. gold, green, hybrid, bronze, diamond or closed.

## `minCitations` (type: `integer`):

Works, authors and institutions modes.

## `hasAbstractOnly` (type: `boolean`):

Works mode.

## `retractedOnly` (type: `boolean`):

Works mode. Lists works that were retracted.

## `sortBy` (type: `string`):

Works, authors and institutions modes. Works can also be sorted by field weighted citation impact.

## `institutionTypes` (type: `array`):

Institutions mode. education, healthcare, company, government, nonprofit, facility, archive or other.

## `minWorks` (type: `integer`):

Authors and institutions modes.

## `minHIndex` (type: `integer`):

Authors and institutions modes.

## `openAlexApiKey` (type: `string`):

Free code from openalex.org. Without it OpenAlex gives a very small daily budget that all users of the same network address share, so works, authors and institutions runs may stop early. arXiv needs no code.

## `contactEmail` (type: `string`):

Sent to OpenAlex as a courtesy contact.

## Actor input object example

```json
{
  "maxItems": 10,
  "mode": "arxiv",
  "searchTerms": [
    "retrieval augmented generation"
  ],
  "searchIn": "everything",
  "arxivSort": "submitted",
  "openAccessOnly": false,
  "hasAbstractOnly": false,
  "retractedOnly": false,
  "sortBy": "citations"
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "searchTerms": [
        "retrieval augmented generation"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("parselab/openalex-arxiv-scholarly-works-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "searchTerms": ["retrieval augmented generation"],
}

# Run the Actor and wait for it to finish
run = client.actor("parselab/openalex-arxiv-scholarly-works-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "searchTerms": [
    "retrieval augmented generation"
  ]
}' |
apify call parselab/openalex-arxiv-scholarly-works-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parselab/openalex-arxiv-scholarly-works-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/AsZ1xd63Lz8GPNhjr/builds/MVSippVRn4kzHSRIv/openapi.json