Scottish Housing Regulator News Publication Index

Build a source-linked news-to-publication index for Scottish social-housing governance teams. The Actor extracts dated SHR news cards, then associates publication and statutory-guidance URLs actually linked in each article's main content. Large site navigation, contacts, quoted commentary and notes-to-editors are excluded.

Output and scope

Rows include newsId (canonical slug), title, url, indexDate/indexDateRaw, sourceCategory "News", updatedDate/updatedDateRaw from the article's explicitly labelled Updated field, relatedPublicationUrls, listingUrl and attribution. publicationDate remains null: neither news-index nor article-updated dates are the underlying publication's date. Publication content and publication dates are not fetched. Related links are limited to official /publications/ and /for-landlords/statutory-guidance/ paths. No summaries, tenant-level information, data endpoint extraction or compliance verdicts.

Source: https://www.housingregulator.gov.scot/about-us/news/

maxItems defaults to 10 (1–20). maxPages defaults to 1 (1–3). Pagination follows only visible sequential official links using ?p=N&s=9 and only when current robots allow them. The source explicitly publishes these query parameters; the Actor never invents other filters or search URLs. It is a bounded current news index, not the complete publication archive. The source category and all date labels are preserved distinctly.

Reuse and provenance

The copyright section of https://www.housingregulator.gov.scot/data-protection/ permits website information reuse excluding logos under the Open Government Licence: https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/ . Observe attribution and accuracy, exclude third-party rights, personal data and logos; no endorsement. Every row carries "Contains public sector information from the Scottish Housing Regulator licensed under the Open Government Licence v3.0", the original URL and dates. Linked documents are not redistributed.

Why this workflow

Inside Housing professionally covers SHR regulatory changes: https://www.insidehousing.co.uk/news/social-landlords-need-certainty-on-net-zero-standards-scottish-housing-regulator-says-97445 . SFHA's 2026 toolkit documents updates reflecting the revised SHR framework: https://www.sfha.co.uk/sites/default/files/2026-03/tracked-changes-to-self-assurance-toolkit-march-2026%29-pdf.pdf . This evidences industry use, not promised sales. Bounded Apify Store searches on 6 October 2026 found no dedicated result for Scottish Housing Regulator or housingregulator.gov.scot. Generic regulatory page monitors remain alternatives; dated index extraction and article-specific publication-link association are this Actor's distinction. Store searches are not exhaustive.

Conservative operation

Serial ordinary HTTPS through standard runtime network egress. No user-selected URLs, configured Apify proxy, rotation, evasion or redirects. Robots fetched first; longest-match rules, wildcard/end-anchor rules and delays respected with minimum two-second spacing. /data/ is excluded. Stop on denial, challenge, 403/429, other HTTP error or layout change; no alternate endpoint or retries. Caps: 32 requests, 2 MB per response, 8 MB total, 480 seconds, 20 items and 3 listing pages. No document downloads. RUN_REPORT reports requests, bytes, pages, emitted count and bounded coverage; errors fail the run and already-emitted data can be partial.

Run and verify

Python 3.12; install requirements.txt, then python main.py. Apify SDK 4.0.2 and BeautifulSoup 4.14.3 pinned; Apify Python Docker image. Run python -m unittest discover -s tests -v. Official 6 October 2026 fixtures test nine listing cards and exactly three associated links in the selected accounting-requirements article, along with schema, date, robots, scope, denial, redirect and caps. See VALIDATION.md for inspected local SDK records.

The 480-second elapsed-time budget is checked between reads and requests. An active socket read can overshoot that budget by up to its 25-second timeout; this is not a hard process-kill deadline.