APRA Regulatory News Scraper

Turn APRA's latest prudential-regulation announcements into structured, source-linked records for regulatory watchlists, compliance triage and consultation tracking. Get publication dates, document types, industry labels, summaries and original consultation-deadline context without manually reviewing every listing card.

What you get

• One record per eligible APRA news announcement, with a stable URL-derived ID.
• Published date in ISO format plus the original source date text.
• APRA document type and industry labels, including their original wording.
• Editorial summary, source paragraphs mentioning consultation closures, and official news/consultation links.
• A deterministic metadata content hash for downstream comparisons and a UTC observation timestamp. The actor does not persist a change history itself.

The actor serves a different workflow from APRA disqualified-person registers and quarterly superannuation datasets. It does not collect those datasets or enrich people. Professional monitoring services such as RegWatch explicitly cover APRA; this supports the monitoring use case, not a revenue claim. Bounded Store searches on 6 October 2026 found no dedicated APRA regulatory-news actor; generic crawlers remain alternatives and Store coverage can change.

Coverage and important limits

Discovery starts at News and publications, in the source's default newest-first order. Only listing cards identified by APRA as news and linking under /news-and-publications/ are fetched. Mixed listing entries, including standalone APRA Explains pages and statistical-publication pages, are excluded and counted in RUN_REPORT. Some excluded statistical pages share the news-and-publications path; the Drupal node type is also checked. This is deliberately a recent-news scraper, not comprehensive APRA publication or archive coverage.

Choose 1–5 listing pages, at most 50 detail fetches and 1–50 matching output records. Filters do not extend this discovery window. Empty results can therefore mean no match within the selected window. Article structure changes fail closed. Missing dates remain null and generate diagnostics.

Deadline context preserves complete source paragraphs, including qualifications such as “Subject to consultation”. It is a best-effort textual convenience, not an exhaustive deadline parser, a computed deadline field, an assertion that an obligation is binding, or legal advice. Check the official source before acting. The content hash covers output metadata, not every word of an article or linked document.

Input

Example:

{
  "maxItems": 10,
  "maxPages": 2,
  "sinceDate": "2026-09-01",
  "keywords": ["trustee", "consultation"],
  "documentTypes": ["Media release"],
  "industries": ["Superannuation"]
}

• maxItems: maximum matching records, default 10, range 1–50.
• maxPages: first source-linked listing pages, default 1, range 1–5.
• sinceDate: optional inclusive publication-date filter in YYYY-MM-DD. Unknown dates are excluded when supplied.
• keywords: optional OR substring matches across title, summary and deadline context, case-insensitive.
• documentTypes: optional OR exact case-insensitive source labels.
• industries: optional OR exact case-insensitive source labels. “All industries” is a distinct source label; it does not automatically match Banking or other individual sectors.

Different filter groups combine with AND. Empty arrays disable their filters. Arbitrary URLs and proxy configuration are not accepted.

Output and run report

Dataset fields: id, source, url, title, documentType, industries, publishedDate, publishedDateText, summary, deadlineContext, officialLinks, contentHash, observedAt, attribution, licenseUrl and diagnostics. The portable contract is dataset.schema.json; .actor/dataset_schema.json configures the Apify dataset view. officialLinks contains label/URL pairs for official news and consultation links; links are not downloaded.

RUN_REPORT in the default key-value store records status, errors, termination reason, requests, response bytes, pages fetched, details checked, records emitted, filter exclusions and non-news listing exclusions. Reaching a configured bound is expected, not proof the archive was exhausted. Any transport, robots or parser error marks the run stopped and fails the Actor; already emitted partial records remain available and must be interpreted with the report.

Responsible access

Uses ordinary Python urllib transport and the environment's normal network path. No rotating proxies, custom proxy routing, credentials, browser impersonation, CAPTCHA handling, retries or redirects. Reads robots.txt first, honors matching allow/disallow and crawl-delay rules, and spaces sequential requests at least two seconds apart. Denials including 401/403/429 immediately stop the run without alternative-route attempts.

Hard request/content limits: 56 total requests including robots, 50 detail pages, 2 MB per response and 30 MB aggregate bytes. A 300-second elapsed-time budget is checked before request launch and after each response, with up to 25 seconds per socket operation. This is not a strict wall-clock kill deadline: a response that continually trickles bytes can overrun the budget before the post-response check. Socket timeouts stop a stalled source rather than retrying it. Only HTTPS www.apra.gov.au, the listing and its bounded page query, and qualified news detail paths can be requested. No attachments, images, registers, site-search endpoints or external links are fetched.

Attribution and reuse

APRA copyright licenses APRA material under CC BY 4.0, subject to exceptions including logos, the Australian Coat of Arms and third-party content. Each record attributes APRA and links its source and license. This project extracts APRA editorial metadata and normalizes dates; no endorsement is implied. Do not extend reuse to excluded third-party material. Fixtures are APRA source pages retained solely for parser tests; their original branding is not part of dataset output.

Local development

python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
python -m unittest discover -s tests -v
mkdir -p storage/key_value_stores/default
printf '{"maxItems":3,"maxPages":1}\n' > storage/key_value_stores/default/INPUT.json
python main.py

Tests use bundled real HTML fixtures and run without network or credentials. TEST_REPORT.md records the local SDK live verification separately from offline tests. No Apify Console deployment or production-container success is claimed by these local checks.