# J-STAGE Scraper (`dadhalfdev/jstage-scraper`) Actor

Scrape Japanese research articles from J-STAGE via the official WebAPI. Use a keyword or paste a J-STAGE URL — the URL wins when set.

- **URL**: https://apify.com/dadhalfdev/jstage-scraper.md
- **Developed by:** [Marco Rodrigues](https://apify.com/dadhalfdev) (community)
- **Categories:**
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🇯🇵 J-STAGE Scraper

Want [J-STAGE](https://www.jstage.jst.go.jp/) Japanese research articles in a spreadsheet? This scraper makes it easy.

Use the **same filters as the global results page** — structured filters work better for AI agents. This actor does **not** take `input_url`; pick a keyword and the sidebar / advanced fields instead.

The public English results UI is [global search](https://www.jstage.jst.go.jp/result/global/-char/en?globalSearchKey=fasting). This actor submits that results form and returns up to 2000 rows per run.

### 💡 Perfect for...

- **Researchers:** Export titles, journals, authors, and years for a topic.
- **Dashboards:** Track new J-STAGE hits without paging the HTML search.
- **Market research:** Slice Japanese scholarly output by subject, access, or year.
- **Data analysts:** Export structured J-STAGE rows with public article URLs.
- **🤖 AI Agents:** Power Claude, Cursor, Codex, the Hermes Agent, and OpenClaw workflows with live J-STAGE results.
- **📚 RAG Systems:** Feed titles, journals, and years into retrieval pipelines.
- **🔗 AI Workflows:** Plug into LangChain, AutoGPT, CrewAI, and similar stacks.

### ✨ Why you'll love this scraper

- 🎯 **Website-Matched Filters:** Keyword, search field, publication type, peer review, article attribute, access type, license, subject, data repository, journal, author, year range, and sort — the same fields as the results form.
- 🌐 **Public result cards:** Title, journal, authors, year, and article URL.

### 📦 What's inside the data?

Every row includes all of these fields:

- **Core:** `id`, `url`, `title`, `journal`
- **People / time:** `authors`, `year`, `doi`

### 🚀 Quick start

1. Type a keyword (`cancer`, `fasting`, …).
2. Optionally narrow with the same filters you would click on the results page.
3. Set `max_results` (up to 2000) and click **Start**.

***

#### Tech details for developers 🧑‍💻

**Input Example:**

```json
{
  "keyword": "fasting",
  "publication_type": "100",
  "peer_reviewed": "1",
  "access_type": "2",
  "subject": "051100",
  "year_from": 2020,
  "year_to": 2024,
  "sort": "5",
  "max_results": 100
}
```

**Output Example:**

```json
{
  "id": "https://www.jstage.jst.go.jp/article/sptgender/21/0/21_41/_article/-char/en",
  "url": "https://www.jstage.jst.go.jp/article/sptgender/21/0/21_41/_article/-char/en",
  "title": "[title in Japanese]",
  "journal": "Journal of Sport and Gender Studies",
  "authors": ["Kari Fasting"],
  "year": "2023",
  "doi": null
}
```

**Output fields:**

| Field | Description |
|-------|-------------|
| `id` | Public article URL. |
| `url` | Public J-STAGE article page. |
| `title` | Title from the result card. |
| `journal` | Publication title. |
| `authors` | Author names from the card. |
| `year` | Published year when shown. |
| `doi` | Not present on the HTML card (always `null`). |

### 📋 Input reference (detailed)

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `keyword` | string | No | `cancer` | Global search box (`freeText` / `globalSearchKey`). |
| `search_field` | enum | No | all fields | Advanced `item1` (title, abstract, author, …). |
| `publication_type` | enum | No | all | `srySybt` (journal, proceedings, …). |
| `peer_reviewed` | enum | No | any | `kjnSdkUm`. |
| `article_attribute` | enum | No | any | `kijiZksi` (last 30 days, AOP, HTML, supplements). |
| `access_type` | enum | No | any | `nnshuSybt` (free / open / authorized). |
| `license` | enum | No | any | `licenseCd` (CC BY family). |
| `subject` | enum | No | all | `bnyaShbnr` subject codes. |
| `data_repository` | enum | No | any | `dataRepo`. |
| `journal` | string | No | — | Journal title facet (`srynm`). |
| `author` | string | No | — | Author facet (`chshnm`). |
| `year_from` / `year_to` | integer | No | — | `yearfrom` / `yearto`. |
| `sort` | enum | No | relevance | `order` / Sort by. |
| `max_results` | integer | No | `50` | Maximum rows (`1`–`2000`). |

#### Notes on filters

- These are the filters on the English global results page, plus the advanced search-field list from the same form.
- There is no `input_url`. Combine filters instead of pasting a results URL.
- Year range and facets only apply when sent with `freeText`, same as the website form.

# Actor input Schema

## `keyword` (type: `string`):

Same as the J-STAGE global search box (`globalSearchKey` / `freeText`). Default is cancer.

## `search_field` (type: `string`):

Optional field for `item1` on the results form. Combined with keyword as `word1` when set.

## `publication_type` (type: `string`):

Same Publication type facet (`srySybt`).

## `peer_reviewed` (type: `string`):

Same Peer review facet (`kjnSdkUm`).

## `article_attribute` (type: `string`):

Same Article attribute facet (`kijiZksi`).

## `access_type` (type: `string`):

Same Access type facet (`nnshuSybt`).

## `license` (type: `string`):

Same License facet (`licenseCd`).

## `subject` (type: `string`):

Same subject facet (`bnyaShbnr`) and advanced-search subject codes.

## `data_repository` (type: `string`):

Same Data repository facet (`dataRepo`).

## `journal` (type: `string`):

Same journal facet (`srynm`). Use the publication title as shown on J-STAGE.

## `author` (type: `string`):

Same author facet (`chshnm`).

## `year_from` (type: `integer`):

Same `yearfrom` field on the results form (YYYY).

## `year_to` (type: `integer`):

Same `yearto` field on the results form (YYYY).

## `sort` (type: `string`):

Same Sort by control (`order` / `sortby`).

## `max_results` (type: `integer`):

How many rows to return (1–2000). The results page can show 50 articles at a time.

## Actor input object example

```json
{
  "keyword": "cancer",
  "search_field": "",
  "publication_type": "",
  "peer_reviewed": "",
  "article_attribute": "",
  "access_type": "",
  "license": "",
  "subject": "",
  "data_repository": "",
  "sort": "1",
  "max_results": 50
}
```

# Actor output Schema

## `overview` (type: `string`):

Table view of scraped articles using the dataset 'overview' view.

## `results` (type: `string`):

All scraped articles from the default dataset without view transformation.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("dadhalfdev/jstage-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("dadhalfdev/jstage-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call dadhalfdev/jstage-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,dadhalfdev/jstage-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iB1ajj8r8feR3PuX7/builds/bszaGBEAMp8geSzJB/openapi.json