# Japan Kokkai Diet Proceedings Scraper - NDL Speech Records (`jungle_synthesizer/kokkai-diet-proceedings-scraper`) Actor Extract speech records from Japan's National Diet Library (NDL) Kokkai API. Search 1M+ speeches across both chambers and all committees (1947–present) by keyword, speaker, committee, or date. Output includes full Japanese speech text, speaker party, Gregorian and wareki dates, and NDL citation URLs. - **URL**: https://apify.com/jungle\_synthesizer/kokkai-diet-proceedings-scraper.md - **Developed by:** [BowTiedRaccoon](https://apify.com/jungle_synthesizer) (community) - **Categories:** Business, Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Japan Diet (Kokkai) NDL Proceedings Scraper Scrapes speech records from [Japan's National Diet Library (NDL) Kokkai API](https://kokkai.ndl.go.jp/api.html). Returns per-speech records across both chambers and all committees from 1947 to the present — over 1 million speeches in the corpus, available at no cost from the NDL's official public API. No auth. No proxy. Pure structured JSON from a government API that actually works. ### Kokkai NDL Scraper Features - **Extracts per-speech records** with full Japanese text, speaker name, party affiliation, official position, and speaker name reading (よみ) - **Covers 75+ years** of Diet proceedings: House of Representatives (衆議院), House of Councillors (参議院), and joint sessions (両院) from 1947 through the current session - **Full-text keyword search** across the entire corpus — Japanese terms, romaji, or policy keywords - **Filters by speaker name**, committee/meeting name, chamber, session number, and date range - **Returns Gregorian and wareki dates** — because your downstream system wants `2026-04-23` and your Japanese colleagues want `令和8年4月23日` - **Stable NDL citation URLs** for every speech and meeting record — suitable for academic references, regulatory citations, and RAG pipelines - **Returns PDF URLs** where available (NDL publishes PDFs with a lag; nulls are normal for recent sessions) - **No proxy required** — the NDL API is a public government service with no IP restrictions ### What Can You Do With Kokkai Proceedings Data? - **Quantitative finance researchers** — track BOJ governor and MOF minister commentary on monetary policy, JGB supply, fiscal consolidation. The Diet record is the unfiltered version. - **Policy researchers** — build comparative parliamentary analysis datasets across sessions, parties, and committees - **LLM training corpora** — formal Japanese diarised speech is rare. This corpus is public domain under Article 13 of Japan's Copyright Law, multi-speaker, and consistently formatted - **Western think tanks** — Brookings, CSIS, and RAND Japan desks spend considerable time translating Diet proceedings. This delivers the raw record programmatically - **Civic tech** — political monitoring, party voting analysis, MP speech frequency dashboards ### How It Works 1. **Configure your search.** Set a keyword, speaker name, committee, chamber, session number, or date range. At least one filter is required — the NDL API doesn't do open-ended dumps. 2. **The scraper calls the NDL speech API** with your filters and paginates through results using the cursor-based `startRecord` / `nextRecordPosition` mechanism. Page size is 100 — the API maximum. 3. **Each speech record is normalized** to the output schema: raw API field names are mapped to snake_case, dates are augmented with wareki equivalents, and speaker position and role are merged into a single `speaker_position` field. 4. **Results are returned** as structured JSON to the Apify dataset. ### Kokkai NDL Scraper Input ```json { "searchQuery": "金融政策", "speakerName": "", "nameOfMeeting": "財務金融委員会", "chamber": "衆議院", "sessionNumber": 0, "dateFrom": "2023-01-01", "dateTo": "2024-12-31", "maxItems": 100 } ```` At least one filter must be set. A search that matches zero results returns zero records rather than an error. | Field | Type | Default | Description | |-------|------|---------|-------------| | `searchQuery` | string | `"予算"` | Full-text keyword search. Supports Japanese and romaji. | | `speakerName` | string | `""` | Filter by speaker name (partial match). E.g., `"安倍"` matches 安倍晋三. | | `nameOfMeeting` | string | `""` | Committee or meeting name filter. E.g., `"予算委員会"`, `"財務金融委員会"`, `"本会議"`. | | `chamber` | string | `""` | Chamber filter: `"衆議院"`, `"参議院"`, `"両院"`, or blank for all. | | `sessionNumber` | integer | `0` | Diet session number (e.g., `213`). `0` = all sessions. | | `dateFrom` | string | `""` | Start date filter (YYYY-MM-DD). Leave blank for earliest (1947-05-20). | | `dateTo` | string | `""` | End date filter (YYYY-MM-DD). Leave blank for most recent. | | `maxItems` | integer | `10` | Maximum records to return. `0` = unlimited. | ### Kokkai NDL Scraper Output ```json { "speech_id": "121104376X01620230425_070", "issue_id": "121104376X01620230425", "session": 211, "chamber": "衆議院", "committee": "財務金融委員会", "issue_number": "第16号", "meeting_date": "2023-04-25", "meeting_date_wareki": "令和5年4月25日", "speech_order": 70, "speaker": "植田和男", "speaker_yomi": "うえだかずお", "speaker_group": "内閣提出", "speaker_position": "日本銀行総裁", "speech_text": "○植田日銀総裁　まず、現在の金融政策運営の考え方についてご説明します...", "speech_url": "https://kokkai.ndl.go.jp/txt/121104376X01620230425/70", "meeting_url": "https://kokkai.ndl.go.jp/txt/121104376X01620230425", "pdf_url": "https://kokkai.ndl.go.jp/pdfb/cm211046_20230425_00.pdf", "search_query": "金融政策", "source_api_endpoint": "https://kokkai.ndl.go.jp/api/speech" } ``` | Field | Type | Description | |-------|------|-------------| | `speech_id` | string | Unique NDL speech identifier | | `issue_id` | string | Meeting record identifier | | `session` | integer | Diet session number (国会回次) | | `chamber` | string | 衆議院, 参議院, or 両院 | | `committee` | string | Committee or meeting name | | `issue_number` | string | Issue label within the session (e.g., 第16号) | | `meeting_date` | string | Meeting date in Gregorian YYYY-MM-DD | | `meeting_date_wareki` | string | Meeting date in Japanese wareki (e.g., 令和5年4月25日) | | `speech_order` | integer | Speaker turn number within the meeting | | `speaker` | string | Speaker full name | | `speaker_yomi` | string | Speaker name reading in hiragana | | `speaker_group` | string | Speaker's party or parliamentary group | | `speaker_position` | string | Official position or role (PM, minister, committee chair, etc.) | | `speech_text` | string | Full speech text in Japanese | | `speech_url` | string | Canonical NDL URL for this speech | | `meeting_url` | string | Canonical NDL URL for the full meeting record | | `pdf_url` | string | PDF URL for the meeting record (null for recent sessions pending publication) | | `search_query` | string | The keyword that returned this record | | `source_api_endpoint` | string | NDL API endpoint that produced this record | ### 🔍 FAQ #### How do I scrape Japan Diet proceedings? The Japan Kokkai NDL Proceedings Scraper calls the National Diet Library's official public API at `kokkai.ndl.go.jp/api/speech`. Set at least one filter — a keyword, speaker name, committee, or date range — and the scraper paginates through all matching records. No credentials or proxy are needed. #### What does the Japan Kokkai Diet Scraper cost to run? The scraper charges $0.10 per run start plus $0.001 per record. A keyword search returning 500 speeches costs roughly $0.60. Unlimited runs returning the full corpus are feasible for budget-conscious users. #### Does the Japan Kokkai Diet Scraper need proxies? The Japan Kokkai NDL Proceedings Scraper doesn't need proxies. The NDL API is a public government service with no IP restrictions or rate limiting beyond reasonable request spacing. #### Can I filter by committee or speaker? The scraper supports filtering by speaker name, committee name, chamber, session number, and date range — independently or in combination. You can pull every BOJ governor speech in the 財務金融委員会 since 2013 in a single run. #### What language is the speech text in? Speech text is in Japanese. The NDL API returns the verbatim Diet record text, which includes standard parliamentary speech conventions (speaker interjections marked ○, procedural text, etc.). Dates are returned in both Gregorian (YYYY-MM-DD) and wareki notation. #### How current is the data? The NDL updates the proceedings database as records are officially published. Major committee hearings from the current session are typically available within days to weeks. PDFs lag longer. *** ### Need More Features? Need a meeting-level endpoint, additional filters, or bulk export by session? [File an issue](https://console.apify.com/actors/issues) or get in touch. ### Why Use Japan Kokkai NDL Proceedings Scraper? - **No competition** — Zero other Apify actors cover the Japanese Diet. You won't find this data pre-packaged anywhere else for $0.001/record. - **Public domain corpus** — Article 13 of Japan's Copyright Law places government records in the public domain. No licensing headaches, no terms-of-service gray zones. - **RAG-ready output** — Each speech is a self-contained chunk with a stable citation URL, speaker attribution, and precise date. Feed it directly into a vector store or policy monitoring pipeline. # Actor input Schema ## `sp_intended_usage` (type: `string`): Please describe how you plan to use the data extracted by this crawler. ## `sp_improvement_suggestions` (type: `string`): Provide any feedback or suggestions for improvements. ## `sp_contact` (type: `string`): Provide your email address so we can get in touch with you. ## `searchQuery` (type: `string`): Full-text keyword to search across speech records (e.g. '予算', '日銀', 'BOJ'). Supports Japanese and romaji. At least one search filter is required. ## `speakerName` (type: `string`): Filter by speaker name (発言者名). Partial match supported (e.g. '安倍' matches 安倍晋三). ## `nameOfMeeting` (type: `string`): Filter by committee or meeting name (委員会名). Examples: 予算委員会, 財務金融委員会, 本会議. ## `chamber` (type: `string`): Filter by chamber. Leave blank for both. ## `sessionNumber` (type: `integer`): Filter by Diet session number (e.g. 213 for the 213th session). Leave blank (0) for all sessions. ## `dateFrom` (type: `string`): Start date filter (Gregorian). Leave blank for earliest available (1947-05-20). ## `dateTo` (type: `string`): End date filter (Gregorian). Leave blank for most recent. ## `maxItems` (type: `integer`): Maximum number of speech records to return. 0 = unlimited. ## Actor input object example ```json { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "searchQuery": "予算", "chamber": "", "sessionNumber": 0, "maxItems": 10 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "searchQuery": "予算", "speakerName": "", "nameOfMeeting": "", "chamber": "", "sessionNumber": 0, "dateFrom": "", "dateTo": "", "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("jungle_synthesizer/kokkai-diet-proceedings-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "searchQuery": "予算", "speakerName": "", "nameOfMeeting": "", "chamber": "", "sessionNumber": 0, "dateFrom": "", "dateTo": "", "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("jungle_synthesizer/kokkai-diet-proceedings-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "searchQuery": "予算", "speakerName": "", "nameOfMeeting": "", "chamber": "", "sessionNumber": 0, "dateFrom": "", "dateTo": "", "maxItems": 10 }' | apify call jungle_synthesizer/kokkai-diet-proceedings-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jungle_synthesizer/kokkai-diet-proceedings-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Japan Kokkai Diet Proceedings Scraper - NDL Speech Records", "description": "Extract speech records from Japan's National Diet Library (NDL) Kokkai API. Search 1M+ speeches across both chambers and all committees (1947–present) by keyword, speaker, committee, or date. Output includes full Japanese speech text, speaker party, Gregorian and wareki dates, and NDL citation URLs.", "version": "0.1", "x-build-id": "elEbuFoajNin8uRWa" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jungle_synthesizer~kokkai-diet-proceedings-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jungle_synthesizer-kokkai-diet-proceedings-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jungle_synthesizer~kokkai-diet-proceedings-scraper/runs": { "post": { "operationId": "runs-sync-jungle_synthesizer-kokkai-diet-proceedings-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jungle_synthesizer~kokkai-diet-proceedings-scraper/run-sync": { "post": { "operationId": "run-sync-jungle_synthesizer-kokkai-diet-proceedings-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sp_intended_usage", "sp_improvement_suggestions" ], "properties": { "sp_intended_usage": { "title": "What is the intended usage of this data?", "minLength": 1, "type": "string", "description": "Please describe how you plan to use the data extracted by this crawler." }, "sp_improvement_suggestions": { "title": "How can we improve this crawler for you?", "minLength": 1, "type": "string", "description": "Provide any feedback or suggestions for improvements." }, "sp_contact": { "title": "Contact Email", "minLength": 1, "type": "string", "description": "Provide your email address so we can get in touch with you." }, "searchQuery": { "title": "Keyword Search", "type": "string", "description": "Full-text keyword to search across speech records (e.g. '予算', '日銀', 'BOJ'). Supports Japanese and romaji. At least one search filter is required." }, "speakerName": { "title": "Speaker Name", "type": "string", "description": "Filter by speaker name (発言者名). Partial match supported (e.g. '安倍' matches 安倍晋三)." }, "nameOfMeeting": { "title": "Committee / Meeting Name", "type": "string", "description": "Filter by committee or meeting name (委員会名). Examples: 予算委員会, 財務金融委員会, 本会議." }, "chamber": { "title": "Chamber", "enum": [ "", "衆議院", "参議院", "両院" ], "type": "string", "description": "Filter by chamber. Leave blank for both.", "default": "" }, "sessionNumber": { "title": "Session Number (国会回次)", "minimum": 0, "type": "integer", "description": "Filter by Diet session number (e.g. 213 for the 213th session). Leave blank (0) for all sessions.", "default": 0 }, "dateFrom": { "title": "Meeting Date From (YYYY-MM-DD)", "type": "string", "description": "Start date filter (Gregorian). Leave blank for earliest available (1947-05-20)." }, "dateTo": { "title": "Meeting Date To (YYYY-MM-DD)", "type": "string", "description": "End date filter (Gregorian). Leave blank for most recent." }, "maxItems": { "title": "Max Items", "minimum": 0, "type": "integer", "description": "Maximum number of speech records to return. 0 = unlimited.", "default": 10 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```