EU Grid Electricity & Gas Procurement Radar

An independent Apify Actor that produces two high-precision procurement signals from the public TED Search API:

• LIVE_TENDER — an open competition with a future submission or expression-of-interest deadline;
• EARLY_EXPIRY_SIGNAL — an awarded contract with a reliable end date approaching the configured watch window.

This is not a generic TED scraper and it does not claim to predict that every expiring contract will be re-tendered.

Deliberately narrow V1

Product	Exact main lot CPV	Rule
Grid electricity	09310000	Supplies contract, EU-27 buyer
Stationary natural gas	09123000	Supplies contract; clear transport/fuel-card signals are excluded, multilingual neutral titles remain eligible

Transport fuel, fuel cards, CNG/biomethane at filling stations, liquid fuels, broad CPV matches, inferred dates, national portals, attachments, OCR, CRM, alerts, and schedules are outside V1.

Exact TED notice types

Stream	Notice types	Behaviour
LIVE	cn-standard, pin-cfc-standard	Accept only a safely mapped lot with a future deadline
EARLY	can-standard	Create or update an awarded-contract watch record
EARLY lifecycle	can-modif, compl	Update or close an existing contract only after a deterministic match

The collector sends ten narrow query families when both CPVs and both streams are enabled: four LIVE searches, two award searches, two modification searches, and two completion searches. It asks TED for latest notice versions only and uses a seven-day overlap on incremental lifecycle scans.

Quality and deduplication

• One output item represents one safely mapped lot.
• Main lot CPV is the inclusion gate; an additional CPV is not enough.
• Ambiguous flattened-array mappings are suppressed by default.
• Contract end dates are A_DIRECT (TED end date) or B_COMPUTED (TED start plus calendar duration). Inferred dates are excluded.
• LIVE identity is procedureId + lotId.
• EARLY identity is procedureId + contractId + lotId; uncertain fallbacks are labelled and review-only.
• Replayed overlap windows are idempotent: unchanged records remain UNCHANGED instead of becoming duplicates.
• TED's production Search API is read with ITERATION pagination, avoiding the 15,000-result page-number ceiling and preserving a consistent point-in-time walk.
• Modification and completion notices that cannot be linked uniquely become anomalies and do not alter another contract.
• Tender-to-expiry linking is LINKED_HIGH only with the same exact buyer identifier, same CPV, and a contract start within 183 days of the old end date. Name-only candidates are review-only.

Input

Field	Default	Meaning
signalTypes	both	Select LIVE, EARLY, or both
cpvCodes	both V1 codes	Electricity, stationary gas, or both
countries	all EU-27	Optional two-letter buyer-country filter
expiryFromDays	90	Start of the early-watch window
expiryToDays	730	End of the early-watch window
includeReview	false	Include clearly labelled ambiguous records for QA
maxItems	1000	Output cap; source lifecycle processing still completes

Output and state

Both streams use one JSON schema. Core fields include signalId, signalType, status, TED identifiers and URL, procedure/lot/contract identity, main CPV, scope decision, buyer, LIVE deadline, EARLY end-date confidence, nullable value, and conservative linkage fields. Under the hard maxItems cap, output alternates LIVE and EARLY records so a busy expiry stream cannot hide current tenders.

Award records beyond the current watch window remain in internal state as PENDING_WINDOW, so they can enter the output later without rescanning the complete history. State is split into 16 deterministic key-value-store shards to avoid one oversized JSON record. The initial eForms backfill starts on 25 October 2023; later scans use per-CPV cursors.

Local verification

Tests use in-memory TED and storage doubles, so they do not contact TED or run the Actor:

$PYTHONPATH=src python -m unittest discover -s tests -v

Current status

The implementation is an independent Apify Actor. Local syntax and unit tests pass. The selected TED fields, query payload, and ITERATION pagination have been checked against production, including real notices 535223-2026, 607260-2026, 611080-2026, 611838-2026, 541608-2025, and 12500-2026. Build 0.0.9 and the first scheduled run succeeded with 1000 dataset records (500 LIVE and 500 EARLY records in the capped output). The daily schedule EU Energy Radar — Daily TED scan is enabled. Source files remain hidden from Store users.