OpenAlex Scholarly Papers & Authors

OpenAlex papers scraper returns scholarly work, author, or institution records with source metadata and available citation fields.

OpenAlex papers scraper uses the public source named below and writes its results to the Actor dataset.

Use it for literature discovery, author and affiliation lookup, citation analysis and research input for AI agents. It reads the official OpenAlex API. OpenAlex metadata is released under CC0.

What it returns

Fields vary by result type and optional enrichment. The examples below are from sample_output.json.

Output field	Type	Example value or excerpt from saved sample
entity_type	string	"work"
status	string	"ok"
query	string	"transformer neural network"
openalex_id	string	"https://openalex.org/W4385245566"
name	string	"Exploiting Generative AI to Scale up Intelligent Tutoring Systems"
api_url	string	"https://api.openalex.org/works/W4385245566"
cited_by_count	integer	78458
errors	array	[]
retrieved_at	string	"2026-10-05T11:33:57.222687+00:00"
source_updated_at	string	"2026-10-03T13:16:28.481748"
title	string	"Exploiting Generative AI to Scale up Intelligent Tutoring Systems"
doi	string	"https://doi.org/10.4230/lipics.itp.2023.19"
publication_year	integer	2023
publication_date	string	"2023-01-01"
work_type	string	"article"
language	string	"en"
abstract	string or null	"As a present to Mizar on its 50th anniversary, we develop an AI/TP system that automatically proves…
authors	array	[{"openalex_id": null}]
institutions	array	[{"openalex_id": "https://openalex.org/I44504214"}]
is_open_access	boolean	true
oa_status	string	"green"
oa_url	string	"https://drops.dagstuhl.de/storage/00lipics/lipics-vol268-itp2023/LIPIcs.ITP.2023.19/LIPIcs.ITP.2023.19.pdf"
landing_page_url	string	"https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ITP.2023.19"
pdf_url	string or null	"https://drops.dagstuhl.de/storage/00lipics/lipics-vol268-itp2023/LIPIcs.ITP.2023.19/LIPIcs.ITP.2023.19.pdf"
source_name	string	"DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)"
source_id	string	"https://openalex.org/S4377196569"
referenced_works	array	[]
concepts	array	[{"id": "https://openalex.org/C41008148"}]
topics	array	[{"id": "https://openalex.org/T10181"}]

Choose works, authors or institutions for each run. Results from multiple queries are deduplicated by OpenAlex ID. Dataset views separate the overview, works, authors, institutions and provenance fields.

Fields	Meaning
entity_type, status, query	Record kind, ok/error and the query that first produced the row
openalex_id, api_url, name	Stable OpenAlex ID, official record URL and display name
cited_by_count	OpenAlex citation count for the selected entity
title, doi, publication_year, publication_date	Work bibliography
work_type, language, source_name, source_id	Work type, language and primary source
abstract	Available inverted-index abstract reconstructed in token order; otherwise null
authors	Work authors with IDs, ORCIDs, positions and institutional affiliations
institutions	Work affiliations or an author's last known institutions from OpenAlex
is_open_access, oa_status, oa_url	Open-access metadata and an available access link
landing_page_url, pdf_url	Public record/PDF links when supplied by OpenAlex
referenced_works, concepts, topics	Referenced work IDs and subject classifications
works_count, h_index, i10_index, two_year_mean_citedness	Author/institution bibliometric indicators when supplied
orcid, works_api_url, counts_by_year	Author identity link, related works query and available annual counts
ror, country_code, institution_type, homepage_url	Institution identity, country, type and homepage
city, region, country, latitude, longitude	Public institution location
retrieved_at, source_updated_at, errors	Retrieval time, source update time and processing errors

Fields vary by entity type; missing optional source values remain null or empty lists. An error row contains its entity type, query, status and error message. An unavailable or malformed abstract does not discard otherwise usable work metadata.

Input example: scholarly works

{
  "entityType": "works",
  "queries": ["transformer neural network"],
  "yearFrom": 2020,
  "yearTo": 2024,
  "conceptIds": ["C154945302"],
  "openAccess": true,
  "minCitations": 100,
  "sortBy": "citations",
  "includeAbstracts": true,
  "maxItems": 8,
  "concurrency": 2
}

Years and minCitations are inclusive. IDs in one filter list use OR; different filters use AND. openAccess: false selects closed-access works; omit the property to include both. Work searches can also filter by topicIds, authorIds and institutionIds. Use IDs such as C154945302, A5108093963, I40120149 or their full OpenAlex URLs.

Concepts are deprecated and frozen, so use topicIds for current classification coverage. Concept filtering is retained for existing workflows.

Author search:

{"entityType":"authors","queries":["Geoffrey Hinton"],"minCitations":100,"maxItems":10}

For author affiliation searches, institutionIds refers to the author's last known institutions. Institutional affiliation here reflects the source's publication metadata, not verified current employment.

Institution search:

{"entityType":"institutions","queries":["University of Oxford"],"countryCode":"GB","institutionType":"education","maxItems":10}

sortBy accepts relevance (default), citations, year (works) or worksCount (authors/institutions). Incompatible filters fail with a field-specific message.

Real output example

Selected fields from the checked local dataset; sample_output.json contains the complete rows:

{
  "entity_type": "work",
  "openalex_id": "https://openalex.org/W4385245566",
  "title": "Exploiting Generative AI to Scale up Intelligent Tutoring Systems",
  "publication_year": 2023,
  "cited_by_count": 78458,
  "doi": "https://doi.org/10.4230/lipics.itp.2023.19",
  "is_open_access": true,
  "status": "ok"
}

Citation counts and other metadata may change after the sample was captured.

Pricing proposal

Custom event	USD per row	USD per 1,000 rows
work	0.0005	0.50
author	0.0005	0.50

Work authors nested in a paper are included in the work price. Institution rows, error rows, duplicates and unmatched searches have no custom event charge. Usable work metadata is billable even when an abstract is unavailable. Apify platform usage and any configured platform start event are separate. The owner must configure the proposed prices before publication; see PRICING.md for dated competitor evidence and setup.

Limits and reliability

• queries: 1–100 non-empty strings, each at most 1,000 characters.
• maxItems: 1–100,000, default 100. This is a shared cap on dataset rows, including errors, across the run.
• concurrency: 1–10 parallel searches, default 2. Requests share a limiter; pages within each search are sequential.
• Each filter-ID list accepts 1–100 IDs. Publication years accept 1–2100.
• Failed network requests, HTTP 429 and server errors get bounded retries with increasing delays. Permanent API errors become uncharged query error rows; malformed records become uncharged item error rows.
• Anonymous OpenAlex access has a smaller daily allowance. An optional own free key can be supplied as the secret apiKey field or the OPENALEX_API_KEY environment variable. Keys are sent in an Authorization header and excluded from result/error messages.
• OpenAlex author disambiguation, affiliation inference and citation counts can contain source errors. Abstracts and OA/PDF links are not available for every work.
• A restarted or migrated run can revisit records: deduplication and the global row counter are scoped to one process. Migration/restart recovery has not been implemented or validated.

FAQ

What limits a search? maxItems caps rows across queries; results are deduplicated by OpenAlex ID. Source quotas can stop a larger search.

Do I need a proxy, login, or API key? Anonymous public access is supported. An optional own OpenAlex API key can increase the source allowance; no proxy is configured.

How is it priced? The proposed pay-per-event model charges work for a unique work and author for a unique author profile. Institution and error rows have no custom event charge.

How are rate limits handled? Workers share a request limiter and retry temporary responses with backoff and Retry-After when present. A source quota failure is reported.

How fresh are citation counts? retrieved_at records this fetch and source_updated_at reports the source’s update time when available. Citation counts can change.

Does it download full papers? No. It returns metadata, available abstracts, and source-provided links.

Use with AI agents / MCP

After publication, agents can discover and invoke this Actor through the Apify MCP server at https://mcp.apify.com. Supply the same JSON input and read the default dataset through the Actor's Results output link.

Example agent instruction: “Find open-access transformer research published from 2020 to 2024, with at least 100 OpenAlex citations. Return eight works with titles, DOIs and available abstracts. Preserve each OpenAlex ID and retrieval date.”

For an author-to-paper workflow, first search authors, choose the intended returned ID, then search works using authorIds. For an institution-to-paper workflow, first search institutions, then pass its ID as institutionIds in a works search.

Local development

Dependencies: apify>=4.0,<5, httpx>=0.28,<1. No package was added to the shared virtual environment.

Run from this actor directory with the provided environment, without the Apify CLI. In PowerShell:

$env:APIFY_LOCAL_STORAGE_DIR = Join-Path (Get-Location) 'storage'
New-Item -ItemType Directory -Force 'storage/key_value_stores/default' | Out-Null
Copy-Item 'example_input.json' 'storage/key_value_stores/default/INPUT.json'
& 'C:/GritWork/earn/.venv/Scripts/python.exe' -m src

Use a fresh storage directory when repeating a run to keep prior datasets separate. The actor also writes a RUN_SUMMARY key-value record containing row counts, errors, API requests, custom event count and elapsed time.

Offline tests:

& 'C:/GritWork/earn/.venv/Scripts/python.exe' -m pytest -m 'not live' -q

Live smoke:

& 'C:/GritWork/earn/.venv/Scripts/python.exe' -m pytest -m live -q

The real-response fixture URLs and capture time are in tests/fixtures/SOURCES.json. Full local and test evidence is in evidence/; scripts/verify_local.py exercises fresh-storage searches, errors and simulated event budgets. scripts/compare_sample.py independently checks saved IDs, bibliography, citation counts and each abstract token position against fresh official API responses. scripts/validate_package.py checks the requested metadata paths and package constraints. Docker configuration uses apify/actor-python:3.12; a Docker build and cloud run remain for the owner to validate before publication.