EU Tenders Scraper & Opportunity Monitor

Status: Phase 0 GO; production Actor implemented and locally verifiable.

This Apify Actor turns public TED procurement notices into deterministic company-fit signals, notice-version correlation, material change detection, and actionable monitoring events. It is not a generic tender scraper: the default dataset contains events that describe what requires action.

The benchmark uses the anonymous official TED API v3 and does not use an LLM, paid upstream API, browser, customer credentials, or proxy.

Run the Actor

Requirements: Node.js 20 or newer. No credentials, cookies, proxy, or API key are used.

npm install
npm run build
npm run lint
npm start

On Apify, configure a schedule using the same monitorId and stateStoreName. The named key-value store carries notice, procedure, pending-update, and deduplication state across runs. Use a distinct monitor ID for every company profile. Set resetState only when intentionally creating a new monitoring baseline.

At least one positive fit signal is required: target CPV, country, NUTS region, buyer term, or capability keyword. Signals add fit points rather than acting as hard filters; for example, a CPV parent match adds 20 points and a target buyer country adds 15, so minimumFitScore 35 requires both while 20 accepts any matching CPV.

The Store's example input (IT services CPV 72*, buyer countries DEU and FRA, a 7-day lookback, 100 scanned notices, at most 10 rows, minimum fit score 20) is a deliberately small demo that finishes in well under a minute. For production monitoring, raise maxScanNotices and maxResults to cover your full profile. Optional expertQuery text uses the official TED expert-query syntax and is combined with the Actor's publication lookback window.

maxScanNotices is an acquisition safety cap; maxResults limits customer-visible rows. The Actor groups positive signals into country, CPV, NUTS, buyer, and capability query shards, scans them in round-robin order, unions them by stable notice-version identity, then ranks all scanned matches. It never describes a capped scan as complete. Check searchTruncated, coverageComplete, totalMatchesObserved, and per-shard metrics in OUTPUT. Unreturned ranked updates remain pending for later runs.

Each dataset row represents one logical update and contains eventTypes[] plus materialChanges[]. Headline event types include:

• NEW_MATCHING_TENDER, NEW_HIGH_PRIORITY_TENDER, FIT_SCORE_INCREASED
• URGENCY_INCREASED, DEADLINE_EXTENDED, DEADLINE_SHORTENED
• MATERIAL_CHANGE_DETECTED, PROCEDURE_LIFECYCLE_UPDATE
• PROCEDURE_CANCELLED, AWARD_PUBLISHED, NO_LONGER_ACTIONABLE

fitScore measures company/tender compatibility and never changes merely because time passed. urgencyScore changes in deterministic deadline bands; priorityScore combines the two. Value ranges require an explicit matching three-letter currency. Eight-digit CPVs are exact; only CPVs ending in * are prefix matches.

Every update includes stable identities, lifecycle classification, source URL, previous scores, and the current normalized notice. Stored open tenders are reevaluated every run even when they are outside lookbackDays, so deadline aging can produce urgency or expiry events without a new TED publication.

Overlapping runs for the same stateStoreName + monitorId are rejected using a leased monitor lock, verified ownership, and a state revision guard. This is the safest practical key-value-store lock, not a database transaction: a platform crash after dataset write but before state commit can still produce an at-least-once duplicate on retry. Do not intentionally schedule overlapping runs. State is bounded by stateRetentionDays and maxStateNotices; any open-state or pending-update pruning is reported explicitly in OUTPUT.

Pricing

This Actor uses Apify Pay Per Event pricing:

• Actionable tender update: $0.02 per dataset row on the Apify Free plan, with lower prices on paid Apify plans (down to $0.014). One row is one consolidated new or changed tender.
• Actor start: $0.01 per GB of run memory (one event per GB, minimum one), charged once per run.

Runs that find nothing new deliver no rows and pay only the start fee. maxResults and your run's maximum total charge cap the number of paid rows; anything not delivered stays pending for the next run. TED data is fetched from the free official TED API, so there are no upstream costs.

When Apify reports PAY_PER_EVENT pricing, each successfully delivered consolidated row is charged exactly once as actionable-tender-update. Rows containing multiple eventTypes[] or materialChanges[] still represent one event. The Actor preflights the remaining event budget, stops before an unpaid row, and retains undelivered rows in pending state for a later run. Under any other pricing model, rows are delivered without a custom PPE charge. The synthetic apify-actor-start event remains platform-managed, and apify-default-dataset-item must not be configured because it would double-charge the default dataset write.

Verify

npm run build
npm test
npm run lint
npm run format:check
npx apify-cli validate-schema

The real-data benchmark writes PHASE0_REPORT.md, benchmark-output/phase0-report.json, benchmark-output/normalized-sample.json, and benchmark-output/amendment-pairs.json. Live results will vary as TED data and service behavior change.

Phase 0 result

The passing run tested 60/60 matrix queries, 4,679 unique notice versions, 100 correlated version pairs, page sizes 25/100/250, ten consecutive pages, resume/repeat behavior, ITERATION continuation, and concurrency 1/3/5/10. See PHASE0_REPORT.md for measured retries, costs, field availability, and limitations. Scores remain transparent deterministic heuristics; no LLM, paid upstream API, credentials, browser, proxy, or HTML scraping is used.