Tadawul Disclosure Monitor — Saudi Exchange

An Apify Actor that monitors official corporate disclosures published by the Saudi Exchange (Tadawul) — Issuer & Financial Advisor Announcements.

It normalizes results into a clean, consistent schema, remembers what it has already returned so scheduled runs only surface new disclosures, and is built with an adapter architecture so more markets could be added later without touching the core logic (this build is intentionally scoped to Saudi Arabia only).

Honesty note about reliability. Tadawul renders its disclosure list client-side with JavaScript. This Actor prefers Tadawul's own structured JSON announcement API when it's reachable, and falls back to a real, headless Chromium browser (via Crawlee's PlaywrightCrawler) to page through the on-page list when more results are needed. It has not been run against the live site from inside this development environment (no outbound network access here), so treat the first run on Apify as a validation run — check the log output, and see "If scraping stops finding results" below if Tadawul has changed its markup/API since this was built.

---

What it does

1. The adapter requests Tadawul's announcement API first, then falls back to a headless-browser pass over the announcement list page if more results are still needed.
2. Every disclosure is normalized into one unified schema (see below).
3. Disclosures are deduplicated against everything this Actor has returned before, using a persistent key-value store — so a daily scheduled run only returns what's new.
4. New (or, if onlyNew is off, all matching) disclosures are pushed to the Actor's Dataset.

---

Input

Field	Type	Description
company	string	Optional company name filter, e.g. "Saudi Aramco"
ticker	string	Optional ticker/symbol filter — Tadawul identifies companies by numeric code, e.g. "2222"
Date	string	Optional lower date bound, DD-MM-YYYY
maxResults	integer	Max disclosures to return (default 100)
onlyNew	boolean	If true (default), only disclosures not seen in a previous run are returned. If false, everything found in the date range is returned, including previously-seen items.
proxyConfiguration	object	Apify Proxy settings. Recommended if Tadawul starts blocking Apify's default IPs.

Example — everything:

{
  "maxResults": 100,
  "onlyNew": true
}

Example — a single company:

{
  "company": "Saudi Aramco",
  "ticker": "2222",
  "maxResults": 50,
  "onlyNew": false
}

A ready-to-use test input is in examples/input-tadawul-test.json — paste it into the Actor's Input tab (or the "Start" JSON editor).

Validation

• If no company or ticker is given, the Actor simply collects general disclosures for the date range.

---

Output (unified schema)

Every row in the Dataset has this shape:

{
  "country": "Saudi Arabia",
  "market": "Tadawul",
  "company_name": "Saudi Aramco",
  "ticker": "2222",
  "disclosure_title": "Saudi Aramco (2222) - Board of Directors' Decisions",
  "disclosure_date": "24-8-2026",
  "disclosure_time": "16:20",
  "disclosure_url": "https://www.saudiexchange.sa/.../announcement/123456",
  "source_data": {}
}

Any field the source doesn't provide is null — nothing is invented. Anything extra the source exposes that doesn't map cleanly to the unified schema is preserved under source_data instead of being discarded.

---

Deduplication

The Actor keeps a persistent "seen" list in a named Apify Key-Value Store (SEEN-DISCLOSURES-TADAWUL), which — unlike the default run-scoped store — survives between runs of the same Actor on Apify. This is what makes onlyNew meaningful across scheduled runs.

The unique ID used for deduplication is chosen in this priority order (per disclosure):

1. the official disclosure ID (Tadawul's anId)
2. the disclosure's own URL
3. its document URL
4. a SHA-256 hash of its core fields (company, ticker, title, type, date, time), as a last resort

See src/utils/deduplication.js.

---

Daily monitoring / scheduling

1. In the Apify Console, open this Actor → Schedules → Create new schedule.
2. Pick a cron expression, e.g. 0 8 * * * for every day at 8:00 AM (Actor's/container's timezone — set your desired timezone on the schedule itself).
3. Attach the input you want (typically onlyNew: true).
4. Each run will then only output disclosures that weren't seen in any previous run.

Viewing results

• Dataset tab on the run: table/JSON/CSV/Excel view of every disclosure pushed in that run (use the Overview view for the key columns).
• Storage → Key-value stores: SEEN-DISCLOSURES-TADAWUL holds the internal "already seen" ledger (not meant for direct consumption, but useful for debugging deduplication).
• Downstream automation (n8n, Telegram, WhatsApp, email, further AI processing) can read the Dataset via the Apify API or an Apify integration.

---

Local development

npm install
# Windows/macOS/Linux with Chromium already available via `playwright install` if needed
apify run   # or: node src/main.js, after setting an input in storage/key_value_stores/default/INPUT.json

On the Apify platform, just create a new Actor from this source (or push with the Apify CLI: apify push) — the Dockerfile uses Apify's official Playwright/Chrome base image, so no manual browser installation is needed.

---

Project structure

.actor/
  actor.json          Actor metadata + Dataset view definition
  INPUT_SCHEMA.json    Input form shown in the Apify Console
src/
  main.js              Entry point: validation, orchestration, summary logging
  adapters/
    base.js            Abstract MarketAdapter contract
    tadawul.js          Tadawul-specific scraping logic
  utils/
    normalizer.js       Maps the adapter's raw output to the unified schema
    deduplication.js    Persistent "seen disclosures" tracking
    dates.js             Date/time parsing helpers
examples/
  input-tadawul-test.json
Dockerfile
package.json
README.md (this file)

Adding another market later (e.g. EGX, ADX, DFM, LSE)

1. Create src/adapters/<market>.js exporting a class that extends MarketAdapter (see src/adapters/base.js) and implements async collect(), returning an array of RawDisclosure objects.
2. Register it and its { country, market } pair in src/main.js.

Nothing else needs to change — normalizeDisclosure(), deduplication, dataset pushing, and logging are all market-agnostic.

---

If scraping stops finding results

Tadawul is a JavaScript-rendered site and its exact markup/API can change over time. If a run logs collected 0 disclosures:

1. Open the run's log and check for the WARNING/ERROR lines from the adapter.
2. Try enabling Apify Proxy (residential) in the input — Tadawul may rate-limit or geo-block data-center IPs.
3. If the site's structure changed, the fix is localized to src/adapters/tadawul.js.

---

Notes on the Apify Store listing

This Actor does not make guarantees like "100% uptime" or "always works" — like any scraper of a third-party website, it depends on the source site's markup/API staying reasonably stable and being reachable from Apify's infrastructure. Please respect Tadawul's terms of use; this Actor only reads publicly published disclosure pages and applies reasonable rate limiting (see defaultCrawlerOptions() in src/adapters/base.js).