# TDnet Timely Disclosures (`stone_jadex/tdnet-timely-disclosures`) Actor

TDnet disclosure list metadata without downloading PDFs.

- **URL**: https://apify.com/stone_jadex/tdnet-timely-disclosures.md
- **Developed by:** [Stone Jade](https://apify.com/stone_jadex) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $25.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## TDnet Timely Disclosures

Apify Actor using Node 20, ESM, Apify SDK v3, got 14 and Cheerio. It reads the public Japanese TDnet disclosure list, preserves Japanese text, and writes one dataset record per disclosure. It resolves PDF and XBRL URLs without downloading either file.

TDnet exposes roughly one month of list history. Older dates can return 404, while weekends and holidays can return empty pages. Those dates are skipped. Schedule daily runs and retain/export their datasets to accumulate longer history. Incremental mode deduplicates recently emitted disclosures across runs; it does not create a historical archive. Today's list can change during the day.

TDnet's captured robots.txt says `User-agent: * / Disallow: /`. The Actor reports this restriction in its log and run summary. It does not enforce Disallow as a stop condition: its requested behavior is to fetch list pages while honoring Crawl-delay. Confirm permission from the site before production operation. This build and validation do not establish that permission.

### Input

```json
{
  "dateFrom": "2026-10-13",
  "dateTo": "2026-10-13",
  "maxItems": 1000,
  "order": "newestFirst",
  "incremental": false,
  "stateKey": "default",
  "companyCode": "4714",
  "proxy": { "useApifyProxy": false }
}
```

Both dates independently default to today in JST (UTC+9), inclusive. Use real YYYY-MM-DD dates; reversed ranges and invalid dates fail before requests. Future dates are skipped without a list request. Missing older dates are skipped after the first page response. Very wide past ranges therefore make one request per unavailable day; use the available month window when possible.

`order` defaults to `newestFirst` (dateTo down to dateFrom); `oldestFirst` visits dateFrom up to dateTo. Rows within a day always retain site order.

`maxItems` defaults to 1000 and caps emitted disclosures after filtering. `companyCode` is optional: five characters match exactly, and four characters match a TDnet code ending in 0. Codes such as `293A0` remain strings. `securitiesCodes` optionally accepts a list of four-character strings (for example `["4714", "293A", "587A"]`); it matches the first four characters of every TDnet code, including a nonzero fifth character. `categories` optionally accepts a list of category names below. Empty lists mean no restriction. Values within each list are ORed, and all supplied filters are combined with AND. Both filters apply before `maxItems` and incremental deduplication; a page with no matches still continues pagination. No proxy is used by default; the standard Apify proxy input supports `useApifyProxy`, groups, country and custom `proxyUrls`.

### Scheduled incremental runs

Set `incremental: true`, `stateKey: "default"`, and `order: "newestFirst"` in your scheduled task input. Omit both dates to resolve the current JST day at each run, and schedule after the disclosure period. Fixed dates stay fixed across scheduled runs. To catch missed days, supply an overlapping recent date range (update it through your scheduling integration); there is no automatic lookback input. Set `maxItems` high enough for the expected new disclosures, and inspect `truncatedAt` for unfinished crawls.

Seen state is stored in the **named** Apify key-value store `tdnet-state`, under `stateKey`, rather than the run's default store. The same account/store and key survive separate runs and tasks. Locally it is under `storage/key_value_stores/tdnet-state`; keep that storage directory between runs. Use separate keys for distinct company, securities-code or category filters, or independent streams. `stateKey` defaults to `default` and accepts 1–256 letters, digits, underscores or hyphens. `incremental` defaults to false; ordinary runs neither read nor write seen state.

Keys hash the disclosure's date, time, company code and title. Only successfully pushed rows are remembered; seen rows are never pushed or charged through `pushData`. State retains the current JST day and preceding 34 days, pruned when loaded and persisted after successful page pushes or newest-first completion. Completed newest-first runs also persist a `contiguousCompleted` watermark (oldest completed date, newest date, and last disclosure key), even when nothing new is emitted. Successful page pushes clear that watermark until the requested range finishes, so capped or failed runs continue through seen pages on retry. Legacy state without a watermark also scans through seen pages. The watermark is deliberately conservative: it advances only after the full requested range completes. Old disclosures outside that window can be emitted again. Changing a title creates a new key; changing only a PDF URL does not. Fully-seen-page stopping assumes the site's newest-first pages remain ordered: a late addition on an older page may be missed. Run with `oldestFirst` to scan all pages when that matters.

Avoid overlapping runs using the same key: state updates are not transactional across concurrent runs. A successful dataset push followed by a failed state write can produce duplicates on retry. Keep/export each run's dataset separately; persistent seen state is not a combined dataset or a guarantee of complete daily coverage.

### Output

Dataset records contain `securitiesCode4`, `disclosureId`, `category`, `sourcePageUrl`, `date`, `time`, `companyCode`, `companyName`, `title`, absolute `pdfUrl`, nullable absolute `xbrlUrl`, `exchange`, `updateHistory`, and UTC ISO `scrapedAt`. Dates are the JST date encoded in the list URL. Empty update history is an empty string; exchange can contain multiple Japanese market abbreviations, for example 東福. The dataset schema's `fields` is a JSON Schema object with properties, required fields and numeric-compatible unions. Original codes remain unchanged. `securitiesCode4` is the first four characters of the five-character TDnet code, including letters and regardless of the fifth character. `disclosureId` is the PDF filename without `.pdf` (an 18-digit string); its embedded date need not equal the disclosure date. `sourcePageUrl` is the absolute list-page URL where the row appeared. `companyName` and `title` preserve full-width spaces (U+3000), repeated internal whitespace and original text; only ASCII whitespace at their ends is trimmed. Other text fields retain their existing whitespace handling. The schema accepts `updateHistory` as string, number or null; this parser emits strings, including an empty string for a blank history cell.

The default key-value store contains `OUTPUT` with the scrape summary (`items`, `truncatedAt`, `datesCovered`, `datesNotReached`, `skippedDates`, and other counters), plus `RUN_SUMMARY` with status, date range, count (`items` and `newItems` are successfully pushed rows), `skippedSeen` (matching seen rows encountered), pages, skipped dates, request attempts, robots restriction, and timestamps. `datesCovered` lists dates whose first page was attempted, including empty/missing days and a partially crawled cutoff day. `skippedDates` includes missing/empty days and future dates. When the cap is reached, `truncatedAt` records the last emitted disclosure (`date`, `time`, `companyCode`, `title`), logs the cutoff, and `datesNotReached` lists subsequent unvisited dates in crawl order. Without a cutoff, `truncatedAt` is null and `datesNotReached` is empty. Reaching the cap stops immediately, so a cutoff does not prove additional matching disclosures exist. Failed runs write a failure summary and fail the Actor; their dataset may already contain earlier pages. Logs report memory in MiB at start and finish.

### Price

**What 1,000 disclosures cost you: $25.00 in total.** Disclosures are charged at $0.025 per saved record ($25 per 1,000) plus a $0.0025 run-start fee per run, so 1,000 records collected in one run cost $25.0025. For comparison, the main TDnet scraper on Apify (nexgendata) charges $0.05 per record plus a $0.005 run-start fee — $50.005 for the same 1,000 records, about twice as much.

### Truncation and completeness

When `maxItems` stops the crawl, the last pushed disclosure has `truncatedAfter: true` and a `truncation` object containing `reason: "maxItems"`, `maxItems`, `truncatedAt`, `datesCovered`, `datesNotReached`, and `skippedDates`. Other disclosures omit these optional fields. Dataset exports therefore carry the cutoff marker without an extra marker row. `OUTPUT` and `RUN_SUMMARY` retain the run summary.

With the default `newestFirst` order, a capped run can include only part of a cutoff day and omit earlier requested days entirely. `truncatedAt` identifies the last emitted disclosure; `datesNotReached` names unvisited dates. `datesCovered` indicates attempted dates, not complete days. A non-null cutoff means the cap was reached, not proof that more matching rows exist. Raise `maxItems` or continue from the cutoff date (using incremental state to skip already emitted rows) when completeness is needed. With `oldestFirst`, later dates can be omitted instead.

### Title categories

Classification is a deterministic title heuristic, not an official TDnet classification. The first matching rule below wins; unmatched titles become `other`. English patterns are case-insensitive. Titles are preserved unchanged. Mixed subjects receive one category according to this precedence:

| Priority | Category | Title patterns |
| --- | --- | --- |
| 1 | `tenderOffer` | 公開買付, 公開買い付け, “tender offer”, standalone TOB |
| 2 | `mergerAcquisition` | 合併, 買収, 会社分割, 株式交換, 株式移転, 株式交付, 子会社化, 事業譲受, 事業譲渡, 資本業務提携, “merger”, “acquisition” |
| 3 | `buyback` | 自己株式 followed by 取得/買付/消却/処分; 自社株買い; “buyback”; “share repurchase” |
| 4 | `forecastRevision` | 業績/通期/四半期/利益/売上 followed by 予想 then 修正; 予想 followed by 上方修正/下方修正; “forecast” and “revision” in either order |
| 5 | `dividend` | 配当, 剰余金の分配, “dividend” |
| 6 | `earnings` | 決算短信, 決算説明, 決算補足, 決算概要, 決算発表, 決算報告, “earnings”, “financial results” |
| 7 | `governance` | コーポレート plus optional one character then ガバナンス; 定款, 役員, 取締役, 監査役, 株主総会, 人事異動; 代表者 followed by 異動; “governance” |
| 8 | `other` | No preceding rule matches |

For example, 業績予想及び配当予想の修正 is `forecastRevision`, while 配当予想の修正 is `dividend`. Heuristics can misclassify ambiguous titles; inspect the original title for downstream decisions.

### Requests and pagination

Requests are sequential with no concurrency, start at least one second apart, and use the larger of that minimum and robots.txt Crawl-delay for the Actor's user agent. got's automatic retry and redirect handling are disabled. 429 and 5xx responses get at most five attempts with exponential backoff (1, 2, 4, 8 seconds), honoring longer Retry-After seconds or HTTP dates. Transport errors and other HTTP errors fail visibly. Robots 404/410 means no available directives; other robots errors fail the run.

List URLs are `https://www.release.tdnet.info/inbs/I_list_001_YYYYMMDD.html`, then 002 and so on (normally 100 disclosures per page). A 404/410 or recognized empty list ends that date; an unexpected HTML page fails rather than silently counting as empty. Filtering alone never ends pagination prematurely. In incremental `newestFirst` mode, a fully seen page ends that day only below the persisted completed watermark date and when the requested newest date includes its completed region; capped or failed runs without a completed watermark scan through seen pages; subsequent dates are still visited. `oldestFirst` continues through seen pages. The item cap ends the run immediately. No scripts, images, PDFs or ZIPs are fetched.

### Local use and verification

```sh
npm ci --cache /tmp/tdnet-npm-cache
npm test
npm run validate
mkdir -p storage/key_value_stores/default
## Put your input JSON into storage/key_value_stores/default/INPUT.json.
npm start
```

`npm run validate` validates all 100 records parsed from the live-captured fixture against `dataset_schema.json.fields` using local Ajv. `npm run validate -- docs/live-run-output.json` validates the fresh live check output. Tests use `node --test` and stored live HTML plus mocked HTTP responses and clocks; tests do not access TDnet. `scripts/live-check.js` fetches fresh robots and list responses with a hard five-request-attempt budget, writes up to five records through the Apify SDK, validates them with Ajv, and records memory/run artifacts. It also inspects up to three additional list pages for nonempty update history within that budget. It is a manual check, not part of the test suite.

The build captured four responses and made one additional list request for validation (five TDnet requests total). See `test/fixtures/README.md`, `docs/tests.log`, `docs/validation.log`, `docs/memory-before.log`, `docs/memory-after.log`, `docs/phase.log` and `docs/run-summary.json`. The supplied Dockerfile uses `apify/actor-node:20`; metadata requests 128 MiB. Measured local RSS is documented in the memory artifacts, rather than a claim that every hosted run fits within that setting. The container itself has not been built in this workspace.

Iteration 4 used five live requests and Ajv-validated five newly emitted records; all four October 13 list pages had blank update history. The nonempty-history fixture uses an actual captured disclosure row with a synthetic history link/value, explicitly documented in `test/fixtures/README.md`. Evidence: `docs/iteration4-live-check.json`, `docs/iteration4-live.log`, `docs/iteration4-tests.log`, and `docs/iteration4-validation.log`. Existing incremental keys from earlier versions may emit whitespace-affected titles once again because the corrected original title changes their hash.

# Actor input Schema

## `dateFrom` (type: `string`):

YYYY-MM-DD, defaults to today in Japan.

## `dateTo` (type: `string`):

Inclusive YYYY-MM-DD, defaults to today in Japan.

## `maxItems` (type: `integer`):

Maximum disclosures

## `companyCode` (type: `string`):

Optional exact five-character TDnet code or four-character security code (matches trailing 0).

## `proxy` (type: `object`):

No proxy by default. Supports Apify proxy or custom proxyUrls.

## `order` (type: `string`):

Visit dates newest first by default; preserve site order within each day.

## `incremental` (type: `boolean`):

Skip disclosures previously pushed under the same stateKey in the named tdnet-state store.

## `stateKey` (type: `string`):

Isolated incremental stream; use a distinct key for different company filters.

## `securitiesCodes` (type: `array`):

Optional four-character security codes, including alphanumeric codes; matches the first four characters regardless of fifth character. Empty means all.

## `categories` (type: `array`):

Optional rule-based title categories. Empty means all; combined with other filters using AND. Allowed values: earnings, dividend, forecastRevision, tenderOffer, mergerAcquisition, governance, buyback, other.

## Actor input object example

```json
{
  "maxItems": 1000,
  "proxy": {
    "useApifyProxy": false
  },
  "order": "newestFirst",
  "incremental": false,
  "stateKey": "default"
}
```

# Actor output Schema

## `results` (type: `string`):

Scraped items in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("stone_jadex/tdnet-timely-disclosures").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("stone_jadex/tdnet-timely-disclosures").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call stone_jadex/tdnet-timely-disclosures --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stone_jadex/tdnet-timely-disclosures"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PIKXFG0Ga6UEC1kF6/builds/eQnG1Lc68zeEV4zKc/openapi.json