HSE Consultation Deadline Scraper

Turn the Health and Safety Executive's consultation hub into deadline records for regulatory trackers and EHS research. Discover recently featured open and closed consultations, or supply known consultation URLs. Each row includes typed opening/closing dates, the source's status, audience/interests and links to supporting documents.

What you get

• One row per recognized consultation, deduplicated by source URL within the run.
• ISO calendar dates, raw deadline-time wording, source URL and retrieval timestamp.
• HSE portal-specific discovery and parsing, including REACH, PFAS and other consultations featured by the hub. This does not read HSE's separate authorisation workbook.
• Dataset exports through Apify in JSON, CSV and Excel; RUN_SUMMARY explains skipped pages, caps and discovery coverage.

Useful for building a review queue or comparing snapshots on your own schedule. The actor does not send alerts, determine compliance obligations, or calculate a legally authoritative deadline.

Quick start

Run with default input to discover the consultation cards shown at https://consultations.hse.gov.uk/. Default output cap: 10 rows. For a narrowly bounded lookup:

{"consultationUrls":["https://consultations.hse.gov.uk/crd-reach/afa-073-01/"],"discoverRecent":false,"maxItems":1,"status":"all"}

consultationUrls: up to 30 plain HTTPS HSE detail URLs with a department and consultation slug. Query strings, fragments, forms, external hosts and document-download URLs cannot be fetched.

discoverRecent: add homepage consultation cards. This is a recent, curated inventory, not a complete historical or current consultation census. Finder pagination and the National Archives are not crawled.

maxItems: 1–30 matching records. No more than 30 detail pages are attempted, even if a filter removes rows.

status: all, open, closed, forthcoming or unknown. Status comes from the source's Opens/Closes/Closed labels, not date arithmetic.

Real source example

Observed from the HSE AFA073-01 page on 5 October 2026:

{
  "title": "Applications for authorisation AFA073-01 - Chromium Trioxide",
  "openedDate": "2026-09-21",
  "closingDate": "2026-11-16",
  "closingTimeText": "closes at 23:59 London time (BST).",
  "observedStatus": "open",
  "audiences": ["Applications for authorisation"],
  "interests": ["Registration, Evaluation, Authorisation & restriction of CHemicals (REACH)"],
  "sourceUrl": "https://consultations.hse.gov.uk/crd-reach/afa-073-01/"
}

The sample omits the long document URLs and run-specific timestamp for readability. Actual rows also include documentUrls, retrievedAt (UTC ISO timestamp), coverage, warnings and attribution. Missing dates become null; warnings identify absent or unrecognized closing dates. The source's BST wording is preserved verbatim, even though the date is in November. No timezone offset is invented.

Coverage and reliability

The homepage currently features selected open and closed consultations. Explicit URLs are visited first. RUN_SUMMARY gives the number of candidates, attempted/fetched pages, output rows, requests, errors and whether candidate processing was truncated. Zero matches do not establish that no consultations exist. Portal redesigns can require parser updates. No cross-run deduplication, lifecycle database, change detection, applicant extraction or document text extraction is provided.

Failures preserve any rows already emitted and mark the run failed. Always inspect RUN_SUMMARY before treating an export as complete. Confirm deadlines on the linked HSE page before acting.

Responsible collection and reuse

Only public metadata and reference links are collected. No response forms, respondent identities, submissions, applicant narrative, logos or document contents are exported. Source HTML is parsed in memory; it is not republished.

The actor checks robots.txt first and fails closed if it cannot obtain valid rules. Requests are sequential, at least five seconds apart, honoring any longer supported crawl delay or request-rate interval. It configures no scraping proxy, credentials, browser evasion or redirects. Standard runtime networking is used. It stops on 401, 403, 429, redirects and source-policy denials, without retries. Each response is limited to 2 MiB, requests have a 20-second socket timeout and reads a 35-second budget; runs have a 32-request and approximately 15-minute collection cap.

Attribution: Health and Safety Executive. Contains public sector information licensed under the Open Government Licence v3.0. See HSE copyright and reuse policy. Third-party rights and logos are excluded from that permission. Supporting documents remain source links; independently review their terms before reusing them. HSE and Delib do not endorse this actor.

Runtime

Python 3.12 with Apify SDK 4.0.2. No browser or third-party parsing dependency is needed. This release is a private, source-specific prototype; source availability and layout may change.