ECB Payments Consultation Scraper

Turn ECB payments consultation pages into a small, structured lifecycle feed: titles, official status, submission deadlines, PDF/document links and published outcome links. Useful for payments compliance teams maintaining consultation calendars and policy inventories.

FREE ORIGINAL SOURCE: The original information is available free of charge on the ECB website. Any fee is for this independent extraction service, not exclusive access to ECB information. This output modifies presentation by extracting and restructuring metadata. It is not an official ECB publication.

Quick start

Run with { "maxItems": 5, "includeDetails": true }. Export the default Apify dataset as JSON or CSV. Nested link arrays are best retained as JSON. RUN_SUMMARY in the default key-value store reports coverage, counts, truncation, completion and errors. Failed runs can contain partial records; check the summary and run status.

Optional consultationUrls selects exact consultation detail URLs found in the supported payments rows of the current catalog. It never expands scope to arbitrary pages. The catalog is always fetched because it owns lifecycle status and deadlines. includeDetails: false performs only robots and catalog requests.

What a record contains

• title, consultationUrl, source, status as stated in the catalog heading
• listDate, with listDateMeaning explicitly unspecified; openingDate remains null because the catalog does not label that date as opening
• deadlineDate (ISO date), deadlineTimeText, deadlineTimezoneText only when explicit; no inferred UTC conversion
• documentLinks, outcomeLinks, correspondingLinks: labeled source links, never downloaded attachments
• detailFetched, detailTitle when enriched, listUrl, observedAt, attribution, free-source notice and license URL

Example observed source row: Appia project, Under review, list date 2026-03-11, deadline 2026-04-22 at 22:00 CET. An expired deadline does not override the source's status. T2 completed consultations include official outcome links. Having no currently open consultations is valid.

Scope and limitations

This MVP covers the current public catalog HTML's payments-title subset (Appia, T2, payment systems, RTGS, TIPS, digital euro, cyber resilience or SIPS) with a primary consultation detail link. It skips PDF-only rows, outcome-only entries, interest-rate consultations, pagination, unseen archive pages and supervisory consultations. At most 20 records; default 5. Explicit URL selection must match that subset. This is not an economic-release calendar, full ECB archive, change-alert service, investment recommendation or legal/financial advice. Source structure can change; inspect error diagnostics rather than assuming zero means no consultations.

Responsible access

Fixed www.ecb.europa.eu HTTPS fetch host, exact catalog/detail paths, standard runtime HTTP transport (including infrastructure egress when configured), no user-selectable target or scraping proxy, no login, no redirects, no retries or access-control workarounds. Reads robots first, conservatively checks wildcard exclusions, and waits at least 5.1 seconds between every request, honoring longer source delays up to 60 seconds (larger delays stop the run). Stops on 401/403/429 or any non-200 response. Caps: 22 requests, 20 records, 2 MiB/page, 25-second request timeout, 240-second elapsed cap checked before each request. A final request may finish after the elapsed cap. No *_content.*.html routes, PDF downloads or submitted-response contents. Run one instance at a time; parallel runs do not coordinate pacing.

Source reuse and attribution

Source: European Central Bank (ECB), https://www.ecb.europa.eu/.

Copyright © for the entire content of this website: European Central Bank, Frankfurt am Main, Germany.

ECB copyright and disclaimer terms govern source material. This Actor preserves a free-original disclosure and modification notice in every record. When offering paid access to these results, show the free-original disclosure before any fee/subscription and at every access. Do not remove those fields. Reuse must be accurate, credit ECB, disclose modifications, and avoid framing ECB pages. Named-author Working/Occasional Papers, logos and third-party works are not reproduced; linked documents remain subject to their own rights. No claim of public-domain status or ECB endorsement.

Code is provided for the owner's private Actor deployment; source content is not relicensed by this project.

Validation

Install requirements.txt, then run python -m unittest -v. Portable fixtures contain the verbatim main elements of official catalog/Appia pages captured with HTTP 200 on 2026-10-05 and the official robots file. Tests cover source status/date/link extraction, scope validation, malformed dates, off-host rejection, redirect refusal and a positive run with the real Apify SDK and local storage using fixture transport. This fixture SDK test does not claim a live Apify deployment. Local live access can depend on the execution network. No paid run or publication is required to run these tests.