# Doaj Open Access Scraper (`kawsar/doaj-open-access-scraper`) Actor

Search open-access articles and journals instantly by subject or keyword. Extract publication titles, publishers, ISSNs, licenses, and DOIs into CSV, JSON, or Excel.

- **URL**: https://apify.com/kawsar/doaj-open-access-scraper.md
- **Developed by:** [Kawsar](https://apify.com/kawsar) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## DOAJ Scraper: Open Access Journals & Articles

<p align="center">
  <img src="https://imgur.com/XxYFTKw.png" alt="Open Access Journals & Articles" style="max-width:100%; border-radius:14px; width:680px;" />
</p>

Search and extract open-access scholarly journals and research articles directly from the public Directory of Open Access Journals (DOAJ) registry. Gather clean bibliographic records—including titles, ISSNs, publishers, licenses, DOIs, and full JSON payloads—into structured datasets ready for instant export as JSON, CSV, or Excel.

Instead of manually navigating web directories, you can collect comprehensive open-access research lists and journal inventories in bulk for literature reviews and academic tracking.

### Use Cases

- **Open access discovery:** Search subjects like climate science or public health to export matching research articles.
- **Journal directory building:** Pull ISSNs, publishers, and licensing terms for active academic journals in a specific discipline.
- **Library & compliance checks:** Keep journal title lists and metadata updated with scheduled recurring runs.
- **License review:** Analyze publishing licenses directly alongside article and journal titles.
- **Free API extractions:** Query public endpoints directly without needing an API account or authentication keys.

### Extracted Data Points

#### Article Records

- Article title and DOI identifier
- Journal title, publisher name, and ISSN values
- Licensing labels (e.g., CC BY)
- Creation and last-updated timestamps
- Complete bibjson metadata payload

#### Journal Records

- Journal titles and publisher metadata
- ISSN and eISSN numbers
- Licensing information and publication flags

### Input Parameters

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `mode` | String | `articles` | Operating mode (`articles` to search research papers, `journals` to search publications). |
| `query` | String | `climate` | Search terms sent to the DOAJ query endpoint. |
| `type` | String | `articles` | Optional alias for `mode` (mode takes precedence if both are set). |
| `maxItems` | Integer | `100` | Maximum number of successful rows to save (up to 1,000). |
| `timeoutSecs` | Integer | `300` | Overall execution limit in seconds. |

#### Example Input

```json
{
  "mode": "articles",
  "query": "climate",
  "maxItems": 100,
  "timeoutSecs": 300
}

```

### Output Format

```json
{
  "last_updated": "2025-08-19T17:13:32Z",
  "bibjson": {
    "title": "Fishing, predation, and temperature drive herring decline in a large marine ecosystem",
    "journal": {
      "title": "Ecology and Evolution",
      "publisher": "Wiley",
      "issns": ["2045-7758"]
    },
    "identifier": [
      { "id": "2045-7758", "type": "eissn" },
      { "id": "10.1002/ece3.8411", "type": "doi" }
    ]
  },
  "id": "000004687a3a411cb466625281f2ceb0",
  "articleTitle": "Fishing, predation, and temperature drive herring decline in a large marine ecosystem",
  "journalTitle": "Ecology and Evolution",
  "publisherName": "Wiley",
  "issnValue": "2045-7758",
  "doiValue": "10.1002/ece3.8411"
}

```

### How It Works

1. Processes configuration inputs, applying search queries and selecting between article or journal modes.
2. Paginates through API results efficiently while automatically skipping duplicate record IDs.
3. Manages connection interruptions smoothly with structured retry intervals.
4. Logs invalid queries or HTTP errors as clean error entries without interrupting execution flow.
5. Generates export-ready datasets available instantly in JSON, CSV, or Excel formats.

### Frequently Asked Questions

**How do I search for journals instead of articles?**

Set the `mode` parameter to `journals`. Article search is used by default.

**What happens if a query yields no results?**

An empty result set logs a completion note without generating error rows, allowing your workflow to finish cleanly.

**Can I export datasets directly to CSV or Excel?**

Yes. Completed datasets can be downloaded instantly in JSON, CSV, or Excel formats from your run overview.

***

### Legal & Compliance

#### Trademark Notice

DOAJ Open Access Journals and related names, logos, and trademarks belong to their respective owners. Their mention in this documentation is strictly to identify data utility and compatibility, and does not imply affiliation, sponsorship, or endorsement.

#### Disclaimer

This tool is an independent utility and is not affiliated with, endorsed by, or officially connected to third-party open-access database providers. Users are responsible for ensuring their usage complies with relevant terms of service, rate limits, and guidelines.

#### Is web scraping legal?

This tool collects publicly available bibliographic records without bypassing access controls. Collecting public data is standard practice, but proper compliance requires considering how the data is used:

- **Personal data:** Ensure dataset usage respects regional privacy regulations (such as GDPR or CCPA) when combining outputs with personal records.
- **Copyright & Terms:** Respect the terms of use, usage limits, and licensing terms (such as Creative Commons CC BY-SA or CC0 dedications) set by data sources.
- **Legal advice:** This summary is provided for general informational purposes and does not constitute formal legal counsel.
  To learn more, read Apify's guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

# Actor input Schema

## `mode` (type: `string`):

articles returns one row per article. journals returns one row per journal.

## `query` (type: `string`):

DOAJ search text, such as climate or title:Ecology. The text is placed in the public search path.

## `type` (type: `string`):

Optional alias for mode: articles or journals. When mode is also set, mode is used.

## `maxItems` (type: `integer`):

Maximum successful rows to save in one run. Values above 1000 are clamped to 1000.

## `timeoutSecs` (type: `integer`):

Stop scheduling new requests after this many seconds. Rows already saved are kept.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds for each call to the DOAJ search API.

## `proxyConfiguration` (type: `object`):

Optional proxy. Default is no proxy. The public DOAJ search API does not need a proxy, and proxies are not used to get around stated limits.

## Actor input object example

```json
{
  "mode": "articles",
  "query": "climate",
  "type": "articles",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetUrl` (type: `string`):

Default dataset for this run.

## `overview` (type: `string`):

Table of DOAJ articles or journals from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "articles",
    "query": "climate",
    "type": "articles",
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("kawsar/doaj-open-access-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "articles",
    "query": "climate",
    "type": "articles",
    "maxItems": 100,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("kawsar/doaj-open-access-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "articles",
  "query": "climate",
  "type": "articles",
  "maxItems": 100,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call kawsar/doaj-open-access-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kawsar/doaj-open-access-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rsDjlIhYOyn8qUmUl/builds/L9l7UIWn5VLWZP5LI/openapi.json