# DOAJ Scraper (`crawlerbros/doaj-scraper`) Actor Scrape the Directory of Open Access Journals (DOAJ), 20K+ peer-reviewed open access journals and 10M+ articles. Search journals by subject/country, search articles by keyword, or look up by ISSN or DOI. - **URL**: https://apify.com/crawlerbros/doaj-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 4 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## DOAJ Scraper Extract open access journal and article metadata from the **Directory of Open Access Journals (DOAJ)** — the world's largest curated index of peer-reviewed open access content, covering **20,000+ journals** and **10 million+ articles** across all academic disciplines. ### What Is DOAJ? DOAJ is a community-curated online directory that indexes high-quality, open access, peer-reviewed journals. It is widely used by researchers, librarians, and publishers to discover legitimate open access venues and verify journal quality. ### What This Actor Does This actor queries the [DOAJ public API](https://doaj.org/api/v4/docs) to: - **Search journals** by keyword, subject area, country, or open-access start year - **Search articles** by keyword, journal name, or year range - **Look up a journal by ISSN** (print or electronic) - **Look up an article by DOI** No authentication or API key is required. The actor respects the 2 requests/second rate limit. ### Modes | Mode | Description | Key Parameters | |------|-------------|----------------| | `searchJournals` | Search peer-reviewed OA journals | `query`, `subjects`, `countries`, `minYear` | | `searchArticles` | Search OA articles | `query`, `journal`, `fromYear`, `toYear` | | `byISSN` | Look up a journal by ISSN | `issn` | | `byDOI` | Look up an article by DOI | `doi` | ### Input Parameters | Parameter | Type | Description | |-----------|------|-------------| | `mode` | Select | Operating mode (required) | | `query` | String | Free-text search query | | `issn` | String | Journal ISSN (e.g. `0099-2240`) | | `doi` | String | Article DOI (e.g. `10.1186/s12866-014-0229-z`) | | `subjects` | String | Subject filter for journal search (e.g. `Medicine`) | | `countries` | String | Country filter for journal search (e.g. `Brazil`) | | `journal` | String | Journal name filter for article search | | `fromYear` | Integer | Filter articles from this year onward | | `toYear` | Integer | Filter articles up to this year | | `minYear` | Integer | Journals with OA start year at or after this year | | `maxItems` | Integer | Maximum records to return (1–1000, default 50) | ### Output #### Journal Fields | Field | Type | Description | |-------|------|-------------| | `doajId` | String | Unique DOAJ identifier | | `title` | String | Journal title | | `alternativeTitle` | String | Alternative title or abbreviation | | `pissn` | String | Print ISSN | | `eissn` | String | Electronic ISSN | | `publisher` | String | Publisher name | | `country` | String | Publisher country | | `language` | Array | Languages published in | | `subjects` | Array | Subject classification terms | | `keywords` | Array | Journal keywords | | `hasApc` | Boolean | Whether article processing charges apply | | `apcMax` | Integer | Maximum APC amount | | `apcCurrency` | String | APC currency code | | `licenseType` | String | License (e.g. `CC BY`) | | `reviewProcess` | Array | Peer review types | | `openAccessStart` | Integer | Year OA publishing began | | `journalUrl` | String | Journal homepage URL | | `recordType` | String | Always `journal` | | `scrapedAt` | String | ISO 8601 timestamp | #### Article Fields | Field | Type | Description | |-------|------|-------------| | `doajId` | String | Unique DOAJ identifier | | `title` | String | Article title | | `journalTitle` | String | Publishing journal name | | `issns` | Array | Journal ISSNs | | `doi` | String | Digital Object Identifier | | `authors` | Array | Author names | | `abstract` | String | Article abstract | | `keywords` | Array | Author-assigned keywords | | `subjects` | Array | Subject classification terms | | `year` | Integer | Publication year | | `month` | Integer | Publication month | | `fulltextUrl` | String | Full-text URL | | `recordType` | String | Always `article` | | `scrapedAt` | String | ISO 8601 timestamp | ### Example Input ```json { "mode": "searchJournals", "query": "medicine", "maxItems": 10 } ```` ```json { "mode": "searchArticles", "query": "covid-19 vaccines", "fromYear": 2021, "toYear": 2023, "maxItems": 20 } ``` ```json { "mode": "byISSN", "issn": "0099-2240" } ``` ### FAQs **Is an API key required?** No. The DOAJ API is fully public and requires no authentication. **How many records can I get?** Set `maxItems` up to 1000 per run. For larger datasets, run the actor multiple times with different queries. **What is the rate limit?** DOAJ allows 2 requests per second. The actor automatically enforces this with a 0.6-second delay between pages. **Can I filter by subject?** Yes — use the `subjects` parameter for journal search to filter by DOAJ subject classification (e.g. `Biology`, `Medicine`, `Engineering`). **What open access licenses are available?** DOAJ covers CC BY, CC BY-SA, CC BY-NC, CC BY-NC-SA, CC BY-ND, CC BY-NC-ND, and other open licenses. **How often is DOAJ updated?** DOAJ continuously updates its index. New journals and articles are added regularly. **Are preprints included?** No. DOAJ only indexes peer-reviewed journals and their articles. # Actor input Schema ## `mode` (type: `string`): What to fetch from DOAJ. ## `query` (type: `string`): Free-text search query (used for searchJournals and searchArticles modes). ## `issn` (type: `string`): Journal ISSN to look up (mode=byISSN). Can be print or electronic ISSN (e.g. 0099-2240). ## `doi` (type: `string`): Article DOI to look up (mode=byDOI). E.g. 10.1186/s12866-014-0229-z ## `subjects` (type: `string`): Narrow journal search by subject area (e.g. 'Biology', 'Medicine'). Passed as part of the query. ## `countries` (type: `string`): Narrow journal search by publisher country (e.g. 'United Kingdom', 'Brazil'). Passed as part of the query. ## `journal` (type: `string`): Narrow article search to a specific journal title (mode=searchArticles). ## `fromYear` (type: `integer`): Filter articles published from this year onward (mode=searchArticles). ## `toYear` (type: `integer`): Filter articles published up to this year (mode=searchArticles). ## `minYear` (type: `integer`): Filter journals with open access start year at or after this year (mode=searchJournals). ## `maxItems` (type: `integer`): Maximum number of records to return. ## Actor input object example ```json { "mode": "searchJournals", "query": "medicine", "maxItems": 50 } ``` # Actor output Schema ## `results` (type: `string`): Dataset containing all scraped DOAJ journals and articles. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "searchJournals", "query": "medicine", "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/doaj-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "searchJournals", "query": "medicine", "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/doaj-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "searchJournals", "query": "medicine", "maxItems": 50 }' | apify call crawlerbros/doaj-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/doaj-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "DOAJ Scraper", "description": "Scrape the Directory of Open Access Journals (DOAJ), 20K+ peer-reviewed open access journals and 10M+ articles. Search journals by subject/country, search articles by keyword, or look up by ISSN or DOI.", "version": "1.0", "x-build-id": "XvcnXS1QurXVMclOc" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~doaj-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-doaj-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~doaj-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-doaj-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~doaj-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-doaj-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "searchJournals", "searchArticles", "byISSN", "byDOI" ], "type": "string", "description": "What to fetch from DOAJ.", "default": "searchJournals" }, "query": { "title": "Search query", "type": "string", "description": "Free-text search query (used for searchJournals and searchArticles modes).", "default": "medicine" }, "issn": { "title": "ISSN", "type": "string", "description": "Journal ISSN to look up (mode=byISSN). Can be print or electronic ISSN (e.g. 0099-2240)." }, "doi": { "title": "DOI", "type": "string", "description": "Article DOI to look up (mode=byDOI). E.g. 10.1186/s12866-014-0229-z" }, "subjects": { "title": "Subject filter (journals)", "type": "string", "description": "Narrow journal search by subject area (e.g. 'Biology', 'Medicine'). Passed as part of the query." }, "countries": { "title": "Country filter (journals)", "type": "string", "description": "Narrow journal search by publisher country (e.g. 'United Kingdom', 'Brazil'). Passed as part of the query." }, "journal": { "title": "Journal filter (articles)", "type": "string", "description": "Narrow article search to a specific journal title (mode=searchArticles)." }, "fromYear": { "title": "From year", "minimum": 1900, "maximum": 2100, "type": "integer", "description": "Filter articles published from this year onward (mode=searchArticles)." }, "toYear": { "title": "To year", "minimum": 1900, "maximum": 2100, "type": "integer", "description": "Filter articles published up to this year (mode=searchArticles)." }, "minYear": { "title": "Min year (journals)", "minimum": 1900, "maximum": 2100, "type": "integer", "description": "Filter journals with open access start year at or after this year (mode=searchJournals)." }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of records to return.", "default": 50 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```