# Crossref Scholarly Works Scraper (`compass_lab/crossref-works-scraper`) Actor Crossref scholarly works scraper and API: export new articles, preprints and books (DOI, title, authors, journal, publisher, dates, citations, licence, corrections) to JSON, CSV or Excel. Filter by search, publisher, journal, type and date. Track new works and citations. - **URL**: https://apify.com/compass_lab/crossref-works-scraper.md - **Developed by:** [COMPASSLAB](https://apify.com/compass_lab) (community) - **Categories:** AI, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $2.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README **Get Crossref Scholarly Works results as clean JSON, CSV or Excel, from the Apify API or on a schedule. No login, about $2.00 for 1,000 results.** ### What does Crossref Scholarly Works Scraper do? **Crossref Scholarly Works Scraper** extracts structured data from **[api.crossref.org](https://api.crossref.org)**. Collect scholarly works from Crossref's public REST API, the registry behind DOIs: articles, preprints, books and proceedings matching a search, DOI prefix (publisher), journal ISSN, type and date range, as clean records (DOI, title, authors, journal, publisher, dates, citation count, licence, subjects, corrections and retractions), watched for new works, citation jumps and corrections. It works as an **API for api.crossref.org data**: run it from Apify Console, on a schedule, or from your own code, and get clean, typed JSON with numbers as numbers and dates in ISO 8601. ### What you get | | | |---|---| | **Data** | 17 fields per item: `doi`, `title`, `authors`, `journal`, `issns`, `publisher`, ... | | **Formats** | JSON, CSV, Excel, HTML, or the Apify API | | **Price** | $2.00 per 1,000 results, pay per result | | **Access** | Public api.crossref.org data only: no login, no cookies, robots.txt respected | | **Licence** | Crossref metadata: 'almost none of the metadata is subject to copyright, and you may use it for any purpose' (Crossref REST API documentation); abstracts are not included | ### Why use Crossref Scholarly Works Scraper? - **Analysts and researchers**: export the data to a spreadsheet or database. - **Developers**: call it from your own code through the Apify API. - **Monitoring**: run it on a schedule and compare results over time. - **AI agents and RAG**: give an LLM live structured data. Main features: - Follows pagination up to `maxPages` pages per start URL and stops at `maxItems` results. - Filters: `query` (search), `doiPrefixes` (doi prefixes), `issns` (journal issns), `types` (work types), `registeredAfter` (registered after), `publishedAfter` (published after), `publishedBefore` (published before), `onlyChanges` (only changes since the last run), so you only get (and pay for) the results you need. - Polite by default: respects robots.txt, at most `maxConcurrency` parallel requests and a delay between requests. - Checks every result against field validators, so layout changes show up as clear data-quality warnings. - Runs on the Apify platform: scheduling, API access, integrations, monitoring and datasets you can export. ### What data can Crossref Scholarly Works Scraper extract? | Field | Type | Description | |---|---|---| | `doi` | string | DOI | | `title` | string | Title | | `authors` | array | Author names, in order | | `journal` | string | Journal, book or proceedings title | | `issns` | array | ISSNs | | `publisher` | string | Publisher | | `type` | string | Work type, e.g. journal-article, posted-content | | `publishedDate` | string | Publication date (YYYY-MM-DD, or year/month when the day isn't given) | | `registeredAt` | ISO 8601 date | First registered at Crossref (ISO 8601) | | `citations` | integer | Times cited (Crossref's is-referenced-by-count) | | `licenseUrl` | string | Licence URL, when deposited | | `subjects` | array | Subjects, when deposited | | `updates` | array | Corrections, retractions or other updates to this work (type: DOI) | | `url` | string | DOI link | | `source` | string | Always 'Crossref' | | `changeType` | string | new, changed, unchanged since the last run with the same input | | `changes` | object | What changed since the last run: {field: {from, to}} | ### How to scrape api.crossref.org 1. Open Crossref Scholarly Works Scraper in Apify Console and go to the Input tab. 2. Enter what to scrape (see the Input section below), for example the start URLs. 3. Set **Max items** to the number of results you need. 4. Click **Start** and wait for the run to finish. 5. Download the results from the Output tab, or fetch them with the API. ### How much will it cost to scrape api.crossref.org? This Actor is priced **per result**: $2.00 per 1,000 results, with no extra charge for platform usage. That is about $2.00 for 1,000 results: 100 results cost $0.20 and 10,000 results cost $20.00. Set a maximum cost per run and the Actor stops when it is reached. ### Input See the Input tab for full configuration options. | Field | Type | Required | Description | |---|---|---|---| | `query` | string | no | Words to search titles, authors and journals for, e.g. 'large language models'. | | `doiPrefixes` | array | no | Only works from these DOI prefixes (publishers), e.g. 10.1038 (Nature). | | `issns` | array | no | Only works in these journals, e.g. 1476-4687. | | `types` | array | no | e.g. journal-article, posted-content (preprints), book-chapter, proceedings-article. | | `registeredAfter` | string | no | YYYY-MM-DD: works first registered at Crossref on or after this date. Default: the last 7 days. | | `publishedAfter` | string | no | YYYY-MM-DD (publication date). | | `publishedBefore` | string | no | YYYY-MM-DD (publication date). | | `onlyChanges` | boolean | no | For scheduled runs: return (and pay for) only new and changed records, compared with the last run with the same input. The snapshot is kept in a named key-value store; delete it to start over. | | `maxItems` | integer | no | Maximum number of items to return (0 = unlimited). | | `maxPages` | integer | no | Maximum listing pages to follow per start URL (pagination). | | `maxConcurrency` | integer | no | Maximum parallel requests (politeness; 1-10). | | `requestDelayMs` | integer | no | Minimum delay between requests, in milliseconds (at least 250). | | `proxyType` | string | no | none (direct connection), datacenter (Apify Proxy, cheapest) or residential (opt-in, billed per GB, fewer blocks). The actor never switches by itself. | | `proxyCountry` | string | no | Two-letter country code for the proxy IP (optional). | Example input: ```json { "query": "large language models", "maxItems": 300, "maxPages": 2, "maxConcurrency": 1, "requestDelayMs": 500, "proxyType": "none" } ``` ### Output You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. Example results from a real run: ```json [ { "doi": "10.56855/j-llans.v5i3.2471", "title": "Nilai-Nilai Moral dalam Peribahasa Nias Masyarakat Hiliserangkai: Kajian Etnolinguistik", "authors": [ "Weywin Sri Nenci Laoli", "Noveri Amal Jaya Harefa", "Iman Sudi Zega", "Noibe Halawa" ], "journal": "Journal of Literature Language and Academic Studies", "issns": [ "2964-2973", "2985-6000" ], "publisher": "Edupedia Publisher", "type": "journal-article", "publishedDate": "2026-08-18", "registeredAt": "2026-09-27T00:06:24+00:00", "citations": 0, "licenseUrl": null, "subjects": [], "updates": [], "url": "https://doi.org/10.56855/j-llans.v5i3.2471", "source": "Crossref", "changeType": "new", "changes": {} }, { "doi": "10.32603/2071-2340-2025-3-65-80", "title": "Перспективы использования языка описания онтологий ONTOL V2 в автоматизации процедуры проверки знаний Prospects for using the ontology description language ONTOL V2 in automating the knowledge verification procedure", "authors": [ "Данил Денисович Пестряков" ], "journal": "Computer Tools in Education", "issns": [ "2071-2359", "2071-2340" ], "publisher": "St. Petersburg Electrotechnical University LETI", "type": "journal-article", "publishedDate": "2025-10-01", "registeredAt": "2026-09-27T01:14:24+00:00", "citations": 0, "licenseUrl": "https://creativecommons.org/licenses/by/4.0", "subjects": [], "updates": [], "url": "https://doi.org/10.32603/2071-2340-2025-3-65-80", "source": "Crossref", "changeType": "new", "changes": {} }, { "doi": "10.32603/2071-2340-2026-1-74-90", "title": "Модели автоматической генерации учебных заданий: сравнительный обзор Models for automatic generation of educational tasks: a comparative analysis", "authors": [ "Дмитрий Сергеевич Бутенко" ], "journal": "Computer Tools in Education", "issns": [ "2071-2359", "2071-2340" ], "publisher": "St. Petersburg Electrotechnical University LETI", "type": "journal-article", "publishedDate": "2026-03-31", "registeredAt": "2026-09-27T01:15:55+00:00", "citations": 0, "licenseUrl": "https://creativecommons.org/licenses/by/4.0", "subjects": [], "updates": [], "url": "https://doi.org/10.32603/2071-2340-2026-1-74-90", "source": "Crossref", "changeType": "new", "changes": {} } ] ``` ### Integrations and API - **Apify API**: start a run and get the results in one HTTP request: ```bash curl -X POST "https://api.apify.com/v2/acts/compass_lab~crossref-works-scraper/run-sync-get-dataset-items?token=" \ -H "Content-Type: application/json" -d '{"query": "large language models", "maxItems": 300}' ``` - **Python** (`pip install apify-client`): ```python from apify_client import ApifyClient client = ApifyClient("") run = client.actor("compass_lab/crossref-works-scraper").call(run_input={"query": "large language models", "maxItems": 300}) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ``` - **JavaScript** (`npm install apify-client`): ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: '' }); const run = await client.actor('compass_lab/crossref-works-scraper').call({"query": "large language models", "maxItems": 300}); const { items } = await client.dataset(run.defaultDatasetId).listItems(); ``` - **Make, Zapier, n8n, Google Sheets, webhooks**: use the Apify integrations (Integrations tab) to send each run's results where you need them, or to start a run from your workflow. - **Schedules**: run it hourly, daily or weekly from Apify Console (Schedules) and always have fresh results. ### Tips and advanced options - Keep **Max items** and **Max pages** as low as you need: fewer pages means a faster, cheaper run. - Raise **Delay between requests** if the site responds slowly; keep **Max concurrency** low to stay polite. - Missing values are `null`. Fields that often come back empty are listed in the run log as data-quality warnings. - **Watch for changes**: every record has `changeType` (`new`, `changed` or `unchanged` compared with the last run with the same input) and `changes` (`{"field": {"from": ..., "to": ...}}`). Schedule the Actor with **Only changes since the last run** on to get, and pay for, only what moved. `new` means new to this input: not returned by an earlier run with the same filters. The last snapshot is kept in a named key-value store called `-watch-` (the run log prints its name); delete it, or change the input, to start over. ### FAQ, disclaimers and support #### Is it legal to scrape api.crossref.org? D91 (Mouad, 2026-10-04) confirmed the scholarly line, covering the missing robots.txt on api.crossref.org. Crossref: 'almost none of the metadata is subject to copyright, and you may use it for any purpose'; abstracts are left out (they may be copyrighted). Polite pool: mailto and User-Agent with contact email, one request at a time (live limit 3/s). The data is published under the **Crossref metadata: 'almost none of the metadata is subject to copyright, and you may use it for any purpose' (Crossref REST API documentation); abstracts are not included**: you may reuse it, including commercially, provided you credit the source as the licence requires. Our Actors are ethical and do not extract any private user data, such as email addresses, gender, or location. They only extract what the user has chosen to share publicly. We therefore believe that our Actors, when used for ethical purposes by Apify users, are safe. However, you should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. #### How many results can I get? Up to `maxItems` per run (0 means no limit), as many as the source lists. Each result is one dataset item, and you are only charged for items that are saved. #### Can I run it on a schedule or from my own code? Yes. Schedule it in Apify Console (Schedules), or call it with the `run-sync-get-dataset-items` endpoint or the Python/JavaScript clients shown in **Integrations and API** above. #### What are the limitations? - None identified so far. #### Where can I get help? Report problems or ideas on the Issues tab. To call this Actor from your own code, see the API tab. ### Related actors the same clean, typed output across sources, so you can combine them in one dataset. | Actor | What it scrapes | Price | |---|---|---| | [Breezy HR Jobs Scraper](https://apify.com/compass_lab/breezy-jobs-scraper) | Job listings from Breezy HR | $1.00 / 1,000 | | [Open Canada Dataset Catalogue Scraper](https://apify.com/compass_lab/canada-general-scraper) | Dataset records from Open Canada | $1.00 / 1,000 | | [Canada Job Bank Jobs Scraper](https://apify.com/compass_lab/canada-jobbank-scraper) | Job listings from Canada Job Bank | $1.00 / 1,000 | | [data.gouv.fr Dataset Catalogue Scraper](https://apify.com/compass_lab/data-gouv-datasets-scraper) | Dataset records from data.gouv.fr | $1.00 / 1,000 | | [Django Weblog Scraper](https://apify.com/compass_lab/django-weblog-scraper) | Posts from Django Weblog | $1.00 / 1,000 | | [Greenhouse Jobs Scraper](https://apify.com/compass_lab/greenhouse-jobs-scraper) | Job listings from Greenhouse | $1.60 / 1,000 | | [Lever Jobs Scraper](https://apify.com/compass_lab/lever-jobs-scraper) | Job listings from Lever | $1.60 / 1,000 | | [Pinpoint Jobs Scraper](https://apify.com/compass_lab/pinpointhq-jobs-scraper) | Job listings from Pinpoint | $1.00 / 1,000 | | [Python Jobs Scraper (python.org)](https://apify.com/compass_lab/python-job-board-scraper) | Job listings from Python Jobs Scraper (python.org) | $2.00 / 1,000 | | [Recruitee Jobs Scraper](https://apify.com/compass_lab/recruitee-jobs-scraper) | Job listings from Recruitee | $1.00 / 1,000 | | [We Work Remotely Jobs Scraper](https://apify.com/compass_lab/weworkremotely-jobs-scraper) | Job listings from We Work Remotely | $2.50 / 1,000 | | [Working Nomads Jobs Scraper](https://apify.com/compass_lab/workingnomads-jobs-scraper) | Job listings from Working Nomads | $2.50 / 1,000 | # Actor input Schema ## `query` (type: `string`): Words to search titles, authors and journals for, e.g. 'large language models'. ## `doiPrefixes` (type: `array`): Only works from these DOI prefixes (publishers), e.g. 10.1038 (Nature). ## `issns` (type: `array`): Only works in these journals, e.g. 1476-4687. ## `types` (type: `array`): e.g. journal-article, posted-content (preprints), book-chapter, proceedings-article. ## `registeredAfter` (type: `string`): YYYY-MM-DD: works first registered at Crossref on or after this date. Default: the last 7 days. ## `publishedAfter` (type: `string`): YYYY-MM-DD (publication date). ## `publishedBefore` (type: `string`): YYYY-MM-DD (publication date). ## `onlyChanges` (type: `boolean`): For scheduled runs: return (and pay for) only new and changed records, compared with the last run with the same input. The snapshot is kept in a named key-value store; delete it to start over. ## `maxItems` (type: `integer`): Maximum number of items to return (0 = unlimited). You pay per result, so this also caps the cost. ## `maxPages` (type: `integer`): Maximum listing pages to follow per start URL (pagination). ## `maxConcurrency` (type: `integer`): Maximum parallel requests (politeness; 1-10). ## `requestDelayMs` (type: `integer`): Minimum delay between requests, in milliseconds (at least 250). ## `proxyType` (type: `string`): none (direct connection), datacenter (Apify Proxy, cheapest) or residential (opt-in, billed per GB, fewer blocks). The actor never switches by itself. ## `proxyCountry` (type: `string`): Two-letter country code for the proxy IP (optional). ## Actor input object example ```json { "query": "large language models", "types": [ "journal-article", "posted-content" ], "onlyChanges": false, "maxItems": 20, "maxPages": 2, "maxConcurrency": 1, "requestDelayMs": 500, "proxyType": "none" } ``` # Actor output Schema ## `dataset` (type: `string`): All scraped items (overview view) # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "large language models", "maxItems": 20, "maxPages": 2, "maxConcurrency": 1, "requestDelayMs": 500, "proxyType": "none" }; // Run the Actor and wait for it to finish const run = await client.actor("compass_lab/crossref-works-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "large language models", "maxItems": 20, "maxPages": 2, "maxConcurrency": 1, "requestDelayMs": 500, "proxyType": "none", } # Run the Actor and wait for it to finish run = client.actor("compass_lab/crossref-works-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "large language models", "maxItems": 20, "maxPages": 2, "maxConcurrency": 1, "requestDelayMs": 500, "proxyType": "none" }' | apify call compass_lab/crossref-works-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,compass_lab/crossref-works-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/dQ2surlYF8HcalOuA/builds/X5vfS1pP42tRXUEdk/openapi.json