# DOAJ Scraper: Open Access Journals & Articles (`arman-bd/doaj-journals-scraper`) Actor

Scrape the Directory of Open Access Journals: journal title, ISSN, publisher, subject, licence, article processing charges and peer-review process. For OA policy analysis.

- **URL**: https://apify.com/arman-bd/doaj-journals-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Developer tools, AI, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 record scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## DOAJ Scraper: Open Access Journals & Articles

![DOAJ Scraper: Journal metadata and per-currency APC pricing, or article abstracts and DOIs, DOAJ's own query syntax passed through](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/doaj-journals-scraper.jpg)

**DOAJ Scraper** pulls vetted open-access journals out of the **Directory of Open Access Journals**, title, ISSN, publisher, country, subject classification, licence, **article processing charges** and peer-review process. Flip one switch and it searches DOAJ's article index instead, returning abstracts, DOIs and full-text links.

APC data is the reason people come here. Very few sources publish article processing charges in structured, per-currency form; DOAJ does, and this Actor hands it to you as a number and a currency code rather than a sentence buried in a PDF.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/doaj-journals-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/doaj-journals-scraper.md
```

### What you get

#### Journals (`resourceType: "journals"`: the default)

| Output field | Meaning |
|---|---|
| `id` | DOAJ internal journal ID |
| `title` | Journal title |
| `issn`, `eissn` | Print ISSN and electronic ISSN (either may be `null`) |
| `publisher`, `country` | Publisher name and its ISO-2 country code |
| `subjects` | Library of Congress subject labels, e.g. `["Physics"]` |
| `language` | Publication languages as ISO codes |
| `license` | Licence types, e.g. `["CC BY", "CC BY-NC-ND"]` |
| `apcAmount`, `apcCurrency` | Headline article processing charge and its currency |
| `apcPrices` | **Every** published price/currency pair, not just the headline one |
| `hasApc` | `true` / `false`, does the journal charge at all |
| `peerReviewProcess` | e.g. `["Anonymous peer review"]` |
| `oaStartYear` | Year the journal became fully open access |
| `url` | The journal's own homepage |
| `lastUpdated` | When DOAJ last revised the record |
| `resourceType`, `query`, `scrapedAt` | Which index, which query produced the row, and the run timestamp |

#### Articles (`resourceType: "articles"`)

`id`, `title`, `doi`, `issn`, `eissn`, `journalTitle`, `publisher`, `country`, `subjects`, `language`, `keywords`, `authors`, `abstract`, `year`, `month`, `volume`, `issue`, `url` (full-text link), `lastUpdated`, plus the same `resourceType` / `query` / `scrapedAt`.

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, any query that failed, and how many queries hit DOAJ's 1000-record API ceiling.

### Common use cases

**1. Compare publishing costs across journals.** Pull everything that charges, then sort on `apcAmount`.

```json
{
 "searchQueries": ["bibjson.apc.has_apc:true AND bibjson.subject.term:\"Medicine\""],
 "maxResults": 1000
}
```

**2. Check OA compliance for a funder mandate.** Funders that require CC BY and no reader-side paywall can be checked directly against `license` and `hasApc`.

```json
{
 "searchQueries": ["bibjson.publisher.name:\"Elsevier\""],
 "subjects": ["Medicine"],
 "maxResults": 500
}
```

**3. Analyse the open-access landscape.** Diamond OA, free to both author and reader, is `hasApc: false`.

```json
{
 "searchQueries": ["bibjson.apc.has_apc:false"],
 "maxResults": 1000
}
```

### Quick start

Simplest possible run:

```json
{
 "searchQueries": ["machine learning"]
}
```

Two subject-filtered queries, capped:

```json
{
 "searchQueries": ["deep learning", "bibjson.subject.term:\"Physics\""],
 "resourceType": "journals",
 "subjects": ["Physics", "Computer science"],
 "maxResults": 300
}
```

Article metadata for a RAG corpus:

```json
{
 "searchQueries": ["climate adaptation"],
 "resourceType": "articles",
 "maxResults": 1000
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `searchQueries` | array | - | **Required.** One search per entry. Plain words or DOAJ field syntax. Each entry is paginated independently, so three queries can return up to 3000 records. |
| `resourceType` | string | `journals` | `journals` or `articles`. One run searches one index, the output shape differs between them. |
| `subjects` | array | `[]` | Client-side filter on the `subjects` field, case-insensitive substring. Empty = keep everything. |
| `maxResults` | integer | `200` | Total cap across all queries. `0` = no cap, but see the 1000-record API ceiling below. |

**Which combinations make sense.** `subjects` is a post-filter, so it can only narrow what a query already returned, if you want subject-scoped results *from the server*, put it in the query itself as `bibjson.subject.term:"Physics"` and DOAJ will do the filtering before pagination. Use the input filter when you want one broad query sliced several ways. `maxResults` is applied across queries in order, so put your most important query first.

#### DOAJ query syntax

| Query | Finds |
|---|---|
| `*` | Everything (capped at 1000) |
| `bibjson.title:"Nature"` | Title match |
| `bibjson.publisher.name:"Elsevier"` | All journals from a publisher |
| `bibjson.apc.has_apc:false` | Journals with no author-side charge |
| `bibjson.subject.term:"Physics"` | Subject-classified journals |
| `issn:2731-3395` | Lookup by ISSN |
| `bibjson.apc.has_apc:true AND bibjson.publisher.country:GB` | Boolean combination |

### Output example

```json
{
 "resourceType": "journal",
 "query": "bibjson.apc.has_apc:true",
 "id": "e5b4b3d3f1a04a.",
 "title": "Communications Engineering",
 "issn": null,
 "eissn": "2731-3395",
 "publisher": "Nature Portfolio",
 "country": "GB",
 "subjects": ["Engineering (General). Civil engineering (General)"],
 "language": ["EN"],
 "license": ["CC BY", "CC BY-NC-ND"],
 "apcAmount": 2290,
 "apcCurrency": "USD",
 "apcPrices": [
 { "price": 1990, "currency": "EUR" },
 { "price": 2290, "currency": "USD" },
 { "price": 1650, "currency": "GBP" }
 ],
 "hasApc": true,
 "peerReviewProcess": ["Anonymous peer review"],
 "oaStartYear": 2022,
 "url": "https://www.nature.com/commseng/",
 "lastUpdated": "2026-01-15T10:33:48Z",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

`RUN_SUMMARY`:

```json
{
 "resourceType": "journals",
 "queriesRequested": 2,
 "queriesFailed": 0,
 "failures": [],
 "recordsSaved": 300,
 "queriesTruncatedByApiCeiling": 1,
 "apiResultCeiling": 1000,
 "filters": {
 "searchQueries": ["deep learning", "bibjson.subject.term:\"Physics\""],
 "resourceType": "journals",
 "subjects": ["Physics"],
 "maxResults": 300
 },
 "finishedAt": "2026-08-06T12:00:04.512Z"
}
```

### Limits and behaviour

- **DOAJ caps the API at 1000 records per query.** Ask for record 1001 and DOAJ answers HTTP 400 and points you at its public data dump. The Actor stops one page short of that boundary, logs a warning naming the query and its true total, and counts it in `RUN_SUMMARY.queriesTruncatedByApiCeiling`. To go deeper, split one broad query into several narrower ones, three subject-scoped queries fetch 3000 records where one broad query fetches 1000. For a full mirror of DOAJ, use their [public data dump](https://doaj.org/docs/public-data-dump/) instead of any API.
- **A bad query doesn't kill the run.** Malformed query syntax gets HTTP 400 from DOAJ; that query is recorded in `RUN_SUMMARY.failures` and the rest continue. The Actor only errors out if *every* query fails.
- **Partial results are kept.** If a query fails on page 4, the 300 records from pages 1-3 stay in the dataset and the failure is still reported.
- **Transient errors are retried.** 429 and 5xx get three attempts with linear backoff. Malformed queries and 404s are treated as fatal immediately, since retrying them cannot help.
- **APC prices are per-currency.** DOAJ publishes a list, so `apcAmount`/`apcCurrency` carry the preferred single price (USD, then EUR, then GBP, then whatever is first) and `apcPrices` carries the complete list. Never compare `apcAmount` across journals without checking `apcCurrency`.
- **Public data only.** No authentication, no personal data, no access-control bypass.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~doaj-journals-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "searchQueries": ["bibjson.subject.term:\"Physics\""],
 "resourceType": "journals",
 "maxResults": 100
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/doaj-journals-scraper').call({
 searchQueries: ['bibjson.apc.has_apc:true'],
 subjects: ['Medicine'],
 maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const charging = items.filter((j) => j.hasApc).sort((a, b) => b.apcAmount - a.apcAmount);
for (const j of charging.slice(0, 10)) {
 console.log(`${j.apcAmount} ${j.apcCurrency}, ${j.title} (${j.publisher})`);
}
```

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need a DOAJ account or API key?** No. You supply no credentials.

**What happens if DOAJ is unavailable?** 429s and 5xx errors are retried with backoff. If a query still fails. It is recorded in `RUN_SUMMARY.failures` and the run continues with the remaining queries.

**Can I schedule it?** Yes. DOAJ records carry `lastUpdated`, so a scheduled run plus a diff on `id` + `lastUpdated` gives you a clean change feed.

**Why is `issn` null on some journals?** Many journals are electronic-only and have no print ISSN. Use `eissn`, one of the two is always present.

**Why is `apcAmount` null when `hasApc` is true?** A handful of DOAJ records flag a charge without publishing the figure. `apcPrices` will be an empty array in that case; the journal's own `url` is the place to look.

**Can I get more than 1000 records for one search?** Not through the API, that's DOAJ's limit, not this Actor's. Split the search into narrower queries, or use DOAJ's public data dump.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `searchQueries` (type: `array`):

One search per entry. Plain words work ('machine learning'). For precision use DOAJ field syntax, e.g. bibjson.subject.term:"Physics", bibjson.publisher.name:"Elsevier", bibjson.apc.has\_apc:false, or \* for everything. Each query is paginated independently.

## `resourceType` (type: `string`):

Search DOAJ's journal index (title, ISSN, publisher, licence, APC, peer review) or its article index (title, authors, abstract, DOI, full-text link). One run searches one index.

## `subjects` (type: `array`):

Keep only records whose DOAJ subject classification contains one of these terms (case-insensitive, matched against the LCC subject labels such as 'Physics' or 'Electronic computers. Computer science'). Leave empty to keep everything the query returned.

## `maxResults` (type: `integer`):

Cap the total records saved across all queries. Set 0 for no cap. note DOAJ's API itself refuses to return more than 1000 records per query, so a broad query is truncated there and the run summary records it.

## Actor input object example

```json
{
  "searchQueries": [
    "machine learning",
    "bibjson.apc.has_apc:false"
  ],
  "resourceType": "journals",
  "subjects": [
    "Physics",
    "Computer science"
  ],
  "maxResults": 200
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "machine learning"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/doaj-journals-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQueries": ["machine learning"] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/doaj-journals-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "machine learning"
  ]
}' |
apify call arman-bd/doaj-journals-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/doaj-journals-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dg1TiNr1t5kjyByyD/builds/I9mqmcyHIgAlIVtDE/openapi.json