TfL Transport Alerts Scraper

Overview

The TfL Transport Alerts Scraper is an Apify Actor that extracts real-time transport alerts and disruption data from Transport for London (TfL) using the official TfL API. Provide an optional keyword filter and the actor returns structured transport alerts including line name, transport mode, status, delay reason, and affected stations. Whether you're monitoring London transport alerts, building commuter dashboards, or conducting urban mobility research, this actor delivers accurate transport alerts data efficiently.

With dual API endpoint querying (Status + Disruptions) and built-in deduplication, the TfL Transport Alerts Scraper ensures comprehensive and clean transport alerts coverage across Tube, Bus, Overground, DLR, and Tram networks.

---

Features

• Multi-Mode Transport Alerts – Covers Tube, Bus, Overground, DLR, and Tram transport alerts in a single run.
• Dual Endpoint Querying – Fetches from both the TfL Status and Disruption APIs to maximize transport alerts coverage.
• Keyword Filtering – Filter transport alerts by any keyword (e.g., "delay", "suspension", "central line").
• Deduplication – Automatically deduplicates transport alerts across both API endpoints to avoid duplicate records.
• Affected Station Extraction – Parses station names from transport alerts delay reasons using regex extraction.
• Proxy Support – Optionally uses Apify residential proxies for reliable transport alerts fetching.
• Error Handling – Logs errors per endpoint and continues processing remaining transport alerts sources.
• Dataset Integration – Automatically pushes all transport alerts to your Apify dataset for easy export.

---

How It Works

1. Input – Provide an optional keyword to filter transport alerts by line, status, or reason.
2. Dual API Fetch – The actor queries both the TfL Status API and Disruption API for all transport modes.
3. Keyword Match – Each transport alert is matched against the keyword across line name, mode, reason, and status fields.
4. Deduplication – A unique ID (line + reason) is used to prevent duplicate transport alerts from both endpoints.
5. Station Extraction – Affected station names are parsed from parentheses in the transport alerts delay reason text.
6. Build Output – Structures all transport alerts into clean records and pushes them to the dataset.

---

Input

Field	Type	Default	Description
keyword	String	""	Optional keyword to filter transport alerts (e.g., "delay", "central"). Leave empty to fetch all transport alerts.
proxyConfiguration	Object	{}	Apify proxy configuration (e.g., { "proxyGroups": ["RESIDENTIAL"] }).

Example input:

{
  "keyword": "delay",
  "proxyConfiguration": {
    "proxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "GB"
  }
}

---

Output

Each transport alert is pushed as a separate dataset record with the following fields:

Field	Type	Description
transport_type	string	Transport mode of the alert (e.g., "tube", "bus", "dlr", "tram").
line	string	Line or service name (e.g., "Central", "Victoria", "System Wide").
status	string	Transport alert status (e.g., "Minor Delays", "Suspended", "Good Service").
delay_reason	string	Full description of the transport alert or disruption reason.
affected_stations	array	List of station names extracted from the transport alert reason text.
city	string	City of the transport alert (always "London").
source	string	Data source (always "TfL API").
timestamp	string	ISO 8601 timestamp of when the transport alert was fetched.

Example output:

{
  "transport_type": "tube",
  "line": "Central",
  "status": "Minor Delays",
  "delay_reason": "Minor delays on the Central line between Liverpool Street (Zone 1) and Stratford (Zone 2/3) due to an earlier signal failure.",
  "affected_stations": ["Zone 1", "Zone 2/3"],
  "city": "London",
  "source": "TfL API",
  "timestamp": "2025-03-22T12:34:56.789Z"
}

---

Transport Modes Covered

The TfL Transport Alerts Scraper covers transport alerts across all major London transport modes:

Mode	Examples
tube	Central, Victoria, Jubilee, Northern, etc.
bus	All London bus routes
overground	London Overground lines
dlr	Docklands Light Railway
tram	London Trams (Croydon)

---

Use Cases

• Commuter Dashboards – Power real-time transport alerts dashboards for London commuters.
• Travel Apps – Integrate live transport alerts into travel planning and navigation apps.
• Urban Mobility Research – Analyze transport alerts frequency and patterns across London lines.
• Disruption Monitoring – Track recurring transport alerts on specific lines or stations.
• Data Journalism – Investigate and report on London transport alerts and reliability trends.
• Smart City Projects – Feed transport alerts data into smart city analytics pipelines.

---

Quick Start

1. Open on Apify – Visit the actor page and click Try for free.
2. Set Keyword (Optional) – Enter a keyword to filter transport alerts (e.g., "suspension", "victoria").
3. Enable Proxy (Optional) – Configure proxy groups for reliable transport alerts fetching.
4. Run the Actor – Start the run and monitor progress in the logs.
5. Download Results – Export the transport alerts dataset as JSON, CSV, or Excel.

---

Technical Stack

• Data Source – TfL Unified API (free, public)
• Endpoints – /Line/Mode/.../Status and /Line/Mode/.../Disruption
• HTTP Client – requests with optional proxy support
• Station Parsing – Regex-based extraction from transport alerts reason text
• Deduplication – Set-based unique alert ID tracking
• Platform – Apify Actor — serverless, scalable, integrated with Dataset and Key-Value Store

---

Related Tools

Actor	Description
Open Street Map Change Tracker	Extracts location and amenity data from OpenStreetMap via the Overpass API.
Product Launch Tracker	Tracks and extracts product launch data from Hacker News.
Amazon Book Scraper	Scrapes book listings, prices, ratings, and metadata from Amazon.
Goodreads Scraper	Extracts book ratings, reviews, and metadata from Goodreads.
SEC EDGAR Scraper	Retrieves company filings (10-K, 10-Q, 8-K) from SEC EDGAR.
Ultimate Video Info Fetcher	Extracts rich video metadata from any yt-dlp supported URL.
Book Metadata Scraper	Extracts rich book metadata from the Open Library database.

---

Changelog

v1.0.0 – Initial Release

• Real-time transport alerts fetching via TfL Unified API
• Dual endpoint querying (Status + Disruptions)
• Keyword-based transport alerts filtering
• Multi-mode coverage (Tube, Bus, Overground, DLR, Tram)
• Affected station extraction from transport alerts reason text
• Deduplication across both API endpoints
• Proxy configuration support
• Dataset integration with error handling

---

Pricing

• Free for basic usage on Apify (up to certain compute limits).
• Paid plans available for higher volume, priority support, and longer runs.
• Proxy credits consumed if residential proxies are enabled.

---

Support & Feedback

• Issues & Ideas – Open a ticket on the Apify Actor issue tracker.
• Documentation – Visit Apify Docs for platform guides.
• API Notes – This actor uses the public TfL Unified API. No API key is required for basic transport alerts access.

---

Disclaimer: This actor accesses publicly available transport alerts data from the TfL Unified API. Please ensure your usage complies with TfL's terms and conditions. This actor is intended for research and informational purposes only.