# OpenAlex Scraper (`crawlerbros/openalex-scraper`) Actor Scrape OpenAlex the free, open catalog of 250M+ scholarly works, authors, institutions, and concepts. Search papers, authors, or fetch by OpenAlex ID / DOI. Pulls citations, open-access status, abstracts, authorships, journals, topics, and more. - **URL**: https://apify.com/crawlerbros/openalex-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 21 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $1.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## OpenAlex Scraper Scrape OpenAlex — the free, open catalog of 250M+ scholarly works, authors, institutions, and concepts. Search papers, authors, or fetch by OpenAlex ID / DOI / PMID. Pulls citations, open-access status, abstracts, authorships, journals, topics. HTTP-only via the public `api.openalex.org` API. No auth, no proxy, no rate-limit drama (100k req/day in the polite pool). ### What this actor does - **Four modes:** `searchWorks`, `searchAuthors`, `byWorkIds`, `byAuthorIds` - **Universal IDs:** OpenAlex (`W…`, `A…`), DOI, PMID, PMCID, ORCID — all auto-normalized - **Reconstructs abstracts** from OpenAlex's inverted index (zero extra API calls) - **Filters:** publication year range, min citation count, open-access only, work type - **Sorts:** relevance, most cited, newest publication date / year - **Empty fields are omitted** — no nulls reach the dataset ### Output per work - `openalexId`, `doi`, `pmid`, `pmcid`, `magId` — universal IDs - `title`, `publicationDate`, `publicationYear`, `type`, `language` - `citedByCount`, `fwci` (field-weighted citation impact), `hasFulltext` - `isOa`, `openAccessOaUrl`, `openAccessStatus`, `bestOaUrl` - `venue` — `{name, issn_l, publisher, type, isOa, license}` - `authorships[]` — `[{authorId, name, orcid, position, institutions}, ...]` (when `includeAuthorships=true`) - `primaryAuthor` — first author display name (always present scalar) - `concepts[]` — top 10 OpenAlex concept tags (when `includeConcepts=true`) - `abstract` — reconstructed text (when `includeAbstract=true` and OpenAlex has it) - `relevanceScore` — search relevance score (search modes) - `openalexUrl` — canonical link - `recordType: "work"`, `scrapedAt` ### Output per author - `openalexId`, `name`, `orcid` - `worksCount`, `citedByCount` - `lastKnownInstitutions[]` - `hIndex`, `i10Index` - `openalexUrl`, `recordType: "author"`, `scrapedAt` ### Input | Field | Type | Default | Description | |---|---|---|---| | `mode` | string | `searchWorks` | `searchWorks` / `searchAuthors` / `byWorkIds` / `byAuthorIds` | | `searchQuery` | string | `large language models` | For `searchWorks` / `searchAuthors` | | `workIds` | array | – | OpenAlex IDs / DOIs / PMIDs / PMCIDs (for `byWorkIds`) | | `authorIds` | array | – | OpenAlex author IDs / ORCIDs (for `byAuthorIds`) | | `publicationYearMin` | int | – | Drop works before this year | | `publicationYearMax` | int | – | Drop works after this year | | `minCitedBy` | int | – | Drop works with fewer citations | | `openAccessOnly` | bool | `false` | Only emit OA works | | `workType` | string | `any` | `article`/`book`/`preprint`/`review`/`dataset`/etc. | | `sortBy` | string | `relevance_score:desc` | Search ordering | | `includeAbstract` | bool | `true` | Reconstruct abstract from inverted index | | `includeAuthorships` | bool | `true` | Full authorship array | | `includeConcepts` | bool | `true` | Top concept tags | | `userAgentEmail` | string | `apify-actor@noreply.apify.com` | OpenAlex polite-pool email | | `maxItems` | int | `50` | Hard cap (1–10000) | #### Example: top-cited LLM papers from 2024 ```json { "mode": "searchWorks", "searchQuery": "large language models", "publicationYearMin": 2024, "minCitedBy": 50, "sortBy": "cited_by_count:desc", "maxItems": 100 } ```` #### Example: lookup specific papers by DOI ```json { "mode": "byWorkIds", "workIds": [ "10.1145/3442188.3445922", "https://doi.org/10.48550/arXiv.2310.06825", "pmid:25524000" ] } ``` #### Example: all works by an author (Geoffrey Hinton) ```json { "mode": "byAuthorIds", "authorIds": ["A1969205038"], "minCitedBy": 100, "maxItems": 200 } ``` #### Example: open-access ML papers only ```json { "mode": "searchWorks", "searchQuery": "machine learning fairness", "openAccessOnly": true, "workType": "article", "publicationYearMin": 2020 } ``` ### Use cases - **Literature reviews** — bulk-export every paper matching a topic across all disciplines - **Citation tracking** — find the most-cited works on a topic, or all works citing a specific paper - **Author intelligence** — track an author's publication record, h-index, institutional affiliations - **Open-access auditing** — find OA copies of every paper in a reading list - **Topic monitoring** — schedule recurring runs to catch new papers in your area - **Cross-database enrichment** — feed DOIs from arXiv / PubMed / Crossref → enrich with OpenAlex citations ### FAQ **What's OpenAlex?** An open replacement for Microsoft Academic Graph: 250M+ scholarly works, 80M+ authors, free for any use, fully indexed by content+citations. See [openalex.org](https://openalex.org). **Is there a rate limit?** Yes — 100k requests/day in the polite pool (anyone with an email in their User-Agent). The actor sets this header automatically. **Why are abstracts sometimes missing?** OpenAlex omits abstracts when their license doesn't permit redistribution. The actor returns whatever's available; missing abstracts mean the source publisher doesn't allow it. **How does it differ from arXiv / PubMed?** OpenAlex is broader — covers all disciplines, all sources (preprint servers, journals, books, datasets). arXiv only covers preprints in physics/math/CS. PubMed only covers biomedical literature. **What ID formats are accepted?** OpenAlex IDs (`W123…`, `A123…`), full DOI URLs (`https://doi.org/10.1145/...`), bare DOIs (`10.1145/...`), `pmid:N`, `pmcid:N`, and ORCIDs (`0000-0001-…`). **What's `fwci`?** Field-weighted citation impact — a paper's citation count normalized to its field's average. 1.0 = field average, 2.0 = twice field average, etc. Useful for cross-discipline comparison. **Why is `concepts` capped at 10?** OpenAlex assigns dozens of low-confidence concepts per work. We keep the top 10 (already sorted by score) for table display compactness; the full list is in OpenAlex's web UI. **How fresh is the data?** Daily — OpenAlex re-indexes nightly from Crossref, PubMed, ORCID, ROR, etc. # Actor input Schema ## `mode` (type: `string`): What to fetch. ## `searchQuery` (type: `string`): Free-text query. ## `workIds` (type: `array`): OpenAlex work IDs (`W123...`), DOIs, PMIDs, or PMCIDs. Examples: `W2741809807`, `https://doi.org/10.1145/3442188.3445922`, `pmid:25524000`. ## `authorIds` (type: `array`): OpenAlex author IDs (`A123...`) or ORCIDs. ## `venueIds` (type: `array`): OpenAlex source IDs prefixed with `S` (journal / repository / conference). Examples: `S4306400194` (arXiv), `S137773608` (Nature), `S99775433` (Science). Or pass an ISSN like `0028-0836` and it will be resolved. ## `institutionIds` (type: `array`): OpenAlex institution IDs prefixed with `I` (university / lab / company). Examples: `I27837315` (Univ. of Michigan), `I97018004` (Stanford), `I136199984` (Harvard), `I32971472` (MIT), `I114027177` (DeepMind). Or pass a ROR ID URL. ## `publicationYearMin` (type: `integer`): Drop works published before this year. ## `publicationYearMax` (type: `integer`): Drop works published after this year. ## `minCitedBy` (type: `integer`): Drop works with fewer citations than this. ## `openAccessOnly` (type: `boolean`): Only emit works with open-access full-text available. ## `workType` (type: `string`): Optional OpenAlex `type` filter. ## `sortBy` (type: `string`): OpenAlex sort field. ## `includeAbstract` (type: `boolean`): Reconstruct the abstract from OpenAlex's inverted index. Costs zero extra API calls. ## `includeAuthorships` (type: `boolean`): Include the full authorship list (with affiliations). ## `includeConcepts` (type: `boolean`): Include the topic / concept tags assigned by OpenAlex. ## `userAgentEmail` (type: `string`): OpenAlex's polite pool requires a contact email in the User-Agent. Default works for low-volume; for production scraping, supply your own email. ## `maxItems` (type: `integer`): Hard cap on emitted records. ## Actor input object example ```json { "mode": "searchWorks", "searchQuery": "large language models", "workIds": [], "authorIds": [], "venueIds": [], "institutionIds": [], "openAccessOnly": false, "workType": "any", "sortBy": "relevance_score:desc", "includeAbstract": true, "includeAuthorships": true, "includeConcepts": true, "userAgentEmail": "apify-actor@noreply.apify.com", "maxItems": 50 } ``` # Actor output Schema ## `works` (type: `string`): Dataset containing all scraped scholarly works or authors. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "searchWorks", "searchQuery": "large language models", "workIds": [], "authorIds": [], "venueIds": [], "institutionIds": [], "openAccessOnly": false, "workType": "any", "sortBy": "relevance_score:desc", "includeAbstract": true, "includeAuthorships": true, "includeConcepts": true, "userAgentEmail": "apify-actor@noreply.apify.com", "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/openalex-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "searchWorks", "searchQuery": "large language models", "workIds": [], "authorIds": [], "venueIds": [], "institutionIds": [], "openAccessOnly": False, "workType": "any", "sortBy": "relevance_score:desc", "includeAbstract": True, "includeAuthorships": True, "includeConcepts": True, "userAgentEmail": "apify-actor@noreply.apify.com", "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/openalex-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "searchWorks", "searchQuery": "large language models", "workIds": [], "authorIds": [], "venueIds": [], "institutionIds": [], "openAccessOnly": false, "workType": "any", "sortBy": "relevance_score:desc", "includeAbstract": true, "includeAuthorships": true, "includeConcepts": true, "userAgentEmail": "apify-actor@noreply.apify.com", "maxItems": 50 }' | apify call crawlerbros/openalex-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/openalex-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "OpenAlex Scraper", "description": "Scrape OpenAlex the free, open catalog of 250M+ scholarly works, authors, institutions, and concepts. Search papers, authors, or fetch by OpenAlex ID / DOI. Pulls citations, open-access status, abstracts, authorships, journals, topics, and more.", "version": "1.0", "x-build-id": "fdgHEfMzVsJU03Sf1" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~openalex-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-openalex-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~openalex-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-openalex-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~openalex-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-openalex-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "searchWorks", "searchAuthors", "byWorkIds", "byAuthorIds", "byVenueIds", "byInstitutionIds" ], "type": "string", "description": "What to fetch.", "default": "searchWorks" }, "searchQuery": { "title": "Search query (mode=searchWorks / searchAuthors)", "type": "string", "description": "Free-text query.", "default": "large language models" }, "workIds": { "title": "Work IDs (mode=byWorkIds)", "type": "array", "description": "OpenAlex work IDs (`W123...`), DOIs, PMIDs, or PMCIDs. Examples: `W2741809807`, `https://doi.org/10.1145/3442188.3445922`, `pmid:25524000`.", "default": [], "items": { "type": "string" } }, "authorIds": { "title": "Author IDs (mode=byAuthorIds)", "type": "array", "description": "OpenAlex author IDs (`A123...`) or ORCIDs.", "default": [], "items": { "type": "string" } }, "venueIds": { "title": "Venue / source IDs (mode=byVenueIds)", "type": "array", "description": "OpenAlex source IDs prefixed with `S` (journal / repository / conference). Examples: `S4306400194` (arXiv), `S137773608` (Nature), `S99775433` (Science). Or pass an ISSN like `0028-0836` and it will be resolved.", "default": [], "items": { "type": "string" } }, "institutionIds": { "title": "Institution IDs (mode=byInstitutionIds)", "type": "array", "description": "OpenAlex institution IDs prefixed with `I` (university / lab / company). Examples: `I27837315` (Univ. of Michigan), `I97018004` (Stanford), `I136199984` (Harvard), `I32971472` (MIT), `I114027177` (DeepMind). Or pass a ROR ID URL.", "default": [], "items": { "type": "string" } }, "publicationYearMin": { "title": "Publication year (min)", "minimum": 1800, "maximum": 2100, "type": "integer", "description": "Drop works published before this year." }, "publicationYearMax": { "title": "Publication year (max)", "minimum": 1800, "maximum": 2100, "type": "integer", "description": "Drop works published after this year." }, "minCitedBy": { "title": "Min citation count", "minimum": 0, "maximum": 1000000, "type": "integer", "description": "Drop works with fewer citations than this." }, "openAccessOnly": { "title": "Open-access only", "type": "boolean", "description": "Only emit works with open-access full-text available.", "default": false }, "workType": { "title": "Work type", "enum": [ "any", "article", "book", "book-chapter", "dataset", "dissertation", "preprint", "report", "review", "letter", "editorial", "erratum", "standard", "supplementary-materials" ], "type": "string", "description": "Optional OpenAlex `type` filter.", "default": "any" }, "sortBy": { "title": "Sort by", "enum": [ "relevance_score:desc", "cited_by_count:desc", "publication_date:desc", "publication_year:desc" ], "type": "string", "description": "OpenAlex sort field.", "default": "relevance_score:desc" }, "includeAbstract": { "title": "Include abstract", "type": "boolean", "description": "Reconstruct the abstract from OpenAlex's inverted index. Costs zero extra API calls.", "default": true }, "includeAuthorships": { "title": "Include authorships", "type": "boolean", "description": "Include the full authorship list (with affiliations).", "default": true }, "includeConcepts": { "title": "Include topics / concepts", "type": "boolean", "description": "Include the topic / concept tags assigned by OpenAlex.", "default": true }, "userAgentEmail": { "title": "User-Agent contact email", "type": "string", "description": "OpenAlex's polite pool requires a contact email in the User-Agent. Default works for low-volume; for production scraping, supply your own email.", "default": "apify-actor@noreply.apify.com" }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Hard cap on emitted records.", "default": 50 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```