# ORR Rail and Road Consultation Scraper (`vixendata/orr-consultation-lifecycle-scraper`) Actor

Export ORR consultation lifecycle stages, publication and closing dates, summaries and reference document links from recent inventory or supplied detail URLs.

- **URL**: https://apify.com/vixendata/orr-consultation-lifecycle-scraper.md
- **Developed by:** [Vixendata](https://apify.com/vixendata) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 recognized consultation/publication/alert metadata records

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## ORR Rail and Road Consultation Scraper

Turn public Office of Rail and Road consultation pages into structured lifecycle records for rail operators, licensing teams and regulatory intelligence workflows. Compare exports by `sourceUrl` to monitor publication dates, deadlines and authoritative consultation stages without rebuilding a manual URL inventory.

### What this actor covers

- The supported individual consultation links on the first [ORR consultation catalog](https://www.orr.gov.uk/search-consultations) page, optionally supplemented with explicit official detail URLs.
- One row per recognized consultation, with ORR's own status, dates, summary and reference document URLs.
- Explicitly distinguishes the source stage “ORR response to consultation” as `recordType: ORR response to external consultation`.

This is a bounded recent inventory, **not the complete archive**. It does not activate Load More, submit search/filter forms, crawl the general `/search/` route, or treat “Current licence casework consultations” as one consultation. On 5 October 2026 the catalog fixture held 24 rows, including 23 supported detail links and one excluded licence-casework aggregate. Catalog composition may change. It does not fetch PDF contents, respondents, survey forms, personal submissions or third-party sites.

### Inputs

Example: `{ "discoverRecent": true, "maxItems": 10, "status": "all", "publishedSince": "" }`

- `consultationUrls`: up to 50 plain HTTPS `www.orr.gov.uk/search-consultations/slug` URLs; explicit URLs are checked first. No custom host, port, query, encoded path, fragment or arbitrary start URL.
- `discoverRecent` (default true): discover supported detail links from the first catalog HTML page.
- `maxItems` (default 10, maximum 50): maximum matching output rows. Independently, at most 50 detail URLs are checked, so restrictive filters can return fewer rows.
- `status`: `all`, `open`, `reviewing responses`, `conclusions published`, or `orr response to consultation`; exact case-insensitive match to the printed source stage. Use all for unrecognized/new stages. No status is inferred from the current date.
- `publishedSince`: optional inclusive ISO `YYYY-MM-DD`. Missing or unparseable publication dates do not match this filter.

### Output

`title`, `publishedDate`, `closingDate`, `publishedDateRaw`, `closingDateRaw`, `status`, `recordType`, `summary`, `topics`, `breadcrumbs`, `documentUrls`, `responseSummaryUrls`, `sourceUrl`, `retrievedAt`, `warnings`, `attribution`.

Dates are ISO calendar dates parsed from the visible ORR labels, not promises of a particular deadline time. Raw labels remain available. Missing dates become null; parse failures add warnings. Topics are read only when an explicit topic field exists; empty means not provided in the recognized detail HTML. Breadcrumbs are navigation context, not inferred topics. Summary comes from the source description metadata. Document links are deduplicated official `/sites/default/files/` file references and are never downloaded. `responseSummaryUrls` is a conservative subset whose anchor explicitly mentions a consultation/response summary or conclusions; it may be empty even where other response material exists.

A verified detail fixture is [ORR discussion document: modifying non-GBR licences in the context of rail reform](https://www.orr.gov.uk/search-consultations/orr-discussion-document-modifying-non-gbr-licences-context-rail-reform): published 2026-07-27, closing 2026-09-18, source stage “Reviewing responses”, one official PDF. Keeping that source stage matters: a passed deadline is not a concluded policy outcome.

`RUN_SUMMARY` reports records, detail pages fetched, requests, errors and coverage. Errors preserve already emitted rows but fail the run as incomplete. A successful zero-result run means no records matched within the checked subset, not that ORR has no consultations.

### Source access and limits

Read-only direct HTTPS, one request at a time, minimum two seconds between request starts (increased for robots crawl-delay/request-rate), maximum 52 total requests including robots/catalog, maximum 50 detail pages, 3 MiB per response, 25-second socket timeout, bounded response read time and a 20-minute request-start budget. Robots delay above 60 seconds fails closed. The actor reads and checks robots before pages; invalid/unavailable robots stops the run. HTTP 401/403/429 stop immediately, without retries. Redirects stop the run. The standard runtime network transport is retained, including infrastructure egress when required by the host environment. No actor-configured scraper proxies, browser fallback, challenge solving, credentials or access-control bypass.

### Reuse and attribution

Source: [Office of Rail and Road terms and conditions](https://www.orr.gov.uk/terms-and-conditions). Contains public sector information from the Office of Rail and Road licensed under the [Open Government Licence v3.0](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). ORR logos and identified third-party material are excluded. This actor exports factual metadata and ORR-authored description text; linked documents and responses retain their own rights and are not republished. No affiliation or endorsement is implied. Retain the output attribution when redistributing records.

### Why this source-specific feed

[Compliance.ai's published regulatory coverage](https://www.compliance.ai/available-regulatory-content/) explicitly lists Office of Rail and Road, UK-ORR, Agency_ID 10000058, with Consultation coverage. This is evidence of a commercial regulatory-intelligence use case, not a claim that this actor is used by that company. Apify Store checks on 5 October 2026 found no dedicated matching actor for `ORR consultations` or `orr.gov.uk`; broader Office Rail Road results were inspected and unrelated. Store availability changes and generic page monitoring tools exist. The gap addressed here is source discovery plus typed ORR stages, deadline fields, document references and external-response classification, rather than raw text for manually supplied URLs.

### Run and test

Python 3.12; install pinned requirements: `pip install -r requirements.txt`.

Run portable offline checks from this folder: `python -m unittest discover -s tests -v`.

Run with the Apify SDK: `python main.py`. Local SDK input can be placed in `storage/key_value_stores/default/INPUT.json`. On Apify, supply input in the actor input editor; dataset exports support JSON/CSV/Excel through Apify. The included Dockerfile runs the same entry point.

Fixtures are real source HTML captured on 5 October 2026, stored within `tests/fixtures` so the backup is independently testable. `LIVE_CHECK.json` documents the local live SDK check and its limits. Deployment and private cloud validation are separate from local tests; no public Store publication is implied.

# Actor input Schema

## `consultationUrls` (type: `array`):

Up to 50 public HTTPS www.orr.gov.uk/search-consultations/slug URLs. Explicit URLs run before recent discovery; only this exact host and path are fetched.

## `discoverRecent` (type: `boolean`):

Read supported consultation detail links on the first /search-consultations catalog page only. No Load More pagination, archive coverage, licence-casework aggregate or search forms.

## `maxItems` (type: `integer`):

Stop after this many matching records, from 1 to 50. At most 50 detail URLs are checked regardless of filters.

## `status` (type: `string`):

Case-insensitive exact match against the stage printed by ORR. Dates never override source stage. Select all to retain new or unknown source stages.

## `publishedSince` (type: `string`):

Optional inclusive YYYY-MM-DD publication-date filter. Missing or unparseable dates do not match. Leave empty for no date filter.

## Actor input object example

```json
{
  "consultationUrls": [],
  "discoverRecent": true,
  "maxItems": 10,
  "status": "all",
  "publishedSince": ""
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("vixendata/orr-consultation-lifecycle-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("vixendata/orr-consultation-lifecycle-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call vixendata/orr-consultation-lifecycle-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,vixendata/orr-consultation-lifecycle-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cHWlZRjefsck0W4l6/builds/Z9emabS7kMcSQWMOA/openapi.json