# Orcid Researcher Profile Scraper (`real-estate-api/orcid-researcher-profile-scraper`) Actor ORCID Researcher Profile Scraper is a Python tool designed to extract researcher profiles, employment history, education, grants, and published works from ORCID. It exports structured academic metadata into clean CSV or JSON formats for bibliometric analysis and research mapping. - **URL**: https://apify.com/real-estate-api/orcid-researcher-profile-scraper.md - **Developed by:** [real-estate-api](https://apify.com/real-estate-api) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.99 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## ORCID Scraper – Researcher Profiles and Affiliations

ORCID Scraper – Researcher Profiles and Affiliations

Extract publicly available researcher profile information and institutional affiliations from the ORCID public API. Search researchers by keyword and collect names, public email addresses, ORCID identifiers, and institution names in a structured dataset. The scraper supports JSON, CSV, and Excel exports through Apify. It does not require an API key or login. ### Features - Search researcher profiles using keywords. - Extract ORCID iDs and researcher names. - Collect institutional affiliations from expanded search results. - Optionally retrieve public person records for additional names and public email addresses. - Preserve the source's available profile information. - Skip duplicate ORCID iDs. - Handle pagination, request timeouts, and temporary API errors. - Save request failures as structured error rows. - Support scheduled searches to refresh researcher datasets. ### Use Cases - **Researcher discovery:** Find researchers associated with topics such as climate science and export their names and affiliations. - **Affiliation research:** Collect institution names without manually opening individual profiles. - **Public email discovery:** Enable person-record retrieval to collect email addresses that researchers have made public. - **Research monitoring:** Schedule recurring searches to refresh researcher lists. - **Academic data management:** Export structured profile information for research directories and institutional analysis. ### What Data Does This Actor Extract? Each successful row represents one researcher. Most fields come from the ORCID expanded search endpoint. When `includePerson` is enabled, the scraper also requests the public person record to retrieve additional names and publicly available email addresses. Affiliations remain sourced from the expanded search results. | Field | Type | Description | |---|---|---| | `orcid-id` | String | ORCID identifier; used to skip duplicate records | | `given-names` | String | Researcher's given names | | `family-names` | String | Researcher's family names | | `credit-name` | String or null | Publicly available credit name | | `other-name` | Array | Other names associated with the researcher | | `email` | Array | Public email addresses; often empty | | `institution-name` | Array | Institution names returned by expanded search | | `scrapedAt` | String | UTC timestamp when the row was saved | | `sourceUrl` | String | API request reference associated with the row | | `error` | String | Error details, present when a request fails | When a search request fails, the scraper saves an error row containing `error`, `sourceUrl`, and `scrapedAt`. Other profile fields are omitted from that row. An empty search result is logged without creating an error row. If an individual person-record request fails, the scraper records the error for that researcher and continues with the next result. ### Input Parameters | Parameter | Type | Default | Description | |---|---|---|---| | `mode` | String | `search` | Researcher search mode; returns one row per ORCID iD | | `query` | String | `climate` | Search query sent to the API | | `includePerson` | Boolean | `false` | Also request the public person record for additional names and public emails | | `maxItems` | Integer | `100` | Maximum number of dataset rows, including error rows; hard cap of 1,000 | | `timeoutSecs` | Integer | `300` | Maximum run duration for starting new requests, in seconds | | `requestTimeoutSecs` | Integer | `30` | Timeout for each individual HTTP request, in seconds | ### Example Input ```json { "mode": "search", "query": "climate", "includePerson": false, "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30 } ``` Set `includePerson` to `true` when additional public names and email addresses are needed. ### Example Output ```json { "orcid-id": "0009-0007-7778-9088", "given-names": "Gerald", "family-names": "Kutney", "credit-name": null, "other-name": [], "email": [], "institution-name": [ "Royal Canadian Geographical Society", "University of Toronto" ], "scrapedAt": "2026-10-12T00:00:00+00:00", "sourceUrl": "API_REQUEST_URL" } ``` The example uses a placeholder for the source request URL. Actual records may contain different fields or values depending on the information publicly available from ORCID. ### Pagination, Limits, and Error Handling The scraper uses the ORCID expanded search API with query, page size, and starting offset parameters. - **Pagination:** Starts at offset `0` and advances according to the number of results returned. - **Page size:** Requests up to 50 records per page, or fewer when the remaining item limit is smaller. - **Maximum results:** The dataset is limited to 1,000 rows, including error rows. - **Sequential requests:** Requests run one at a time, with approximately 0.75 seconds between calls. - **Deduplication:** Duplicate ORCID iDs are skipped. - **Run timeout:** New requests stop being initiated when `timeoutSecs` is reached. - **Retry handling:** HTTP 429 and 5xx responses, as well as request timeouts, are retried three additional times with waits of 2, 4, and 8 seconds. - **Retry-After:** A longer wait specified by the response takes precedence. - **Other HTTP errors:** Other 4xx responses are not retried. - **Invalid responses:** HTTP 200 responses containing invalid JSON are handled as errors. - **Failed searches:** A failed expanded-search request produces an error row and stops further pagination. - **Failed person lookups:** A failed person-record request produces an error row for that researcher, and processing continues. - **Empty searches:** A query with no matching results produces a log entry rather than an error row. The scraper is designed for the public API and should remain within ORCID's published rate limits and usage quotas. ### Data Source and Licensing Researcher information is retrieved from the public ORCID API, including expanded search and optional public person records. ORCID makes certain registry data available under a limited licence for public data. That licence does not automatically cover third-party materials linked from a record, such as full-text articles. Users should review the applicable ORCID public API terms before collecting, redistributing, or reusing data. Non-commercial-use restrictions, attribution requirements, and published rate limits may apply. The scraper does not require an API key for the supported public endpoints. Each record includes a `sourceUrl` reference for source identification. ### Frequently Asked Questions **Does this ORCID scraper require an API key?** No. It uses the public expanded search and person-record endpoints without requiring an API key or login. **Why are most email arrays empty?** ORCID only returns email addresses that researchers have made publicly available. An empty array means no public email address was returned for that record. **What does `includePerson` do?** When enabled, the scraper requests the researcher's public person record and uses it to populate available public email addresses, other names, and the credit name. Institutional affiliations continue to come from expanded search results. **Why did I receive an error row?** A request may fail because of an invalid query, an HTTP error, a timeout, or a response that cannot be parsed as JSON. The scraper records the available error details rather than silently treating a failed request as a successful result. **What happens when a query returns no matches?** The scraper logs the empty result and completes without creating a provider record or treating the empty result as an API failure. **Can I export results to CSV, JSON, or Excel?** Yes. Apify provides dataset export options for JSON, CSV, and Excel. **How many rows can one run save?** The default is 100 rows, with a hard maximum of 1,000, including error rows. ### Changelog **Version 1.0** - Added ORCID expanded search with offset-based pagination. - Added optional public person-record retrieval. - Implemented duplicate detection, retries, and error handling. - Set a maximum dataset size of 1,000 rows. ### Trademark Notice ORCID and related names, logos, and trademarks belong to their respective owners. Their use in this documentation identifies the data source and does not imply affiliation, sponsorship, endorsement, or partnership. ### Disclaimer This is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with ORCID or its owners. The scraper collects publicly available information. Users are responsible for complying with applicable API terms, licensing conditions, attribution requirements, rate limits, and privacy laws. ### Is Web Scraping Legal? Accessing publicly available information does not automatically make every use of that information lawful. Legal obligations depend on the data collected, the intended use, and applicable laws and source terms. - **Personal data:** Researcher records may contain names, email addresses, and affiliation details. Collect and store this information only for appropriate purposes and in accordance with applicable privacy laws, including the GDPR or CCPA where relevant. - **Copyright:** Descriptions, articles, and other linked content may be protected by copyright. Check the applicable licence before reproducing or redistributing such material. - **Terms of use:** Respect ORCID's published API terms, usage quotas, rate limits, and licensing restrictions. - **Legal advice:** This section provides general information, not legal advice. Consult a qualified legal professional if you are unsure about your intended use. # Actor input Schema ## `mode` (type: `string`): Search reads the public ORCID expanded search. Each row is one researcher. ## `query` (type: `string`): Search text sent as the q parameter on the public expanded-search endpoint. The default, climate, matches the public search used by this actor. ## `includePerson` (type: `boolean`): When checked, each hit is followed by GET /v3.0/{orcid}/person so public emails, other names, and the credit name can be filled in. Off by default to stay on the search endpoint. ## `maxItems` (type: `integer`): Maximum dataset rows for this run, including error rows. Values above 1000 are clamped to 1000. ## `timeoutSecs` (type: `integer`): Stop starting new ORCID requests after this many seconds. This does not replace the Apify run timeout. ## `requestTimeoutSecs` (type: `integer`): Seconds to wait for a single ORCID HTTP response before retrying. ## `proxyConfiguration` (type: `object`): Optional proxy for ORCID requests. The default sends traffic directly, with no Apify Proxy. Do not use proxies to get around ORCID rate limits. ## Actor input object example ```json { "mode": "search", "query": "climate", "includePerson": false, "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `datasetUrl` (type: `string`): API URL for the default dataset of this run. ## `datasetItemsUrl` (type: `string`): Item list for the default dataset. ## `overviewUrl` (type: `string`): Overview table of researcher profiles. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "query": "climate", "includePerson": false, "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("real-estate-api/orcid-researcher-profile-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "query": "climate", "includePerson": False, "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("real-estate-api/orcid-researcher-profile-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "query": "climate", "includePerson": false, "maxItems": 100, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } }' | apify call real-estate-api/orcid-researcher-profile-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,real-estate-api/orcid-researcher-profile-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/TpxIJnam1hPB1Z2hN/builds/Omj3SgNQwVyHEB5lG/openapi.json