# Changelog of Public Tenders Scraper: EU TED & UK Contracts (`rookdev/public-tenders-monitor`) Actor

- **URL**: https://apify.com/rookdev/public-tenders-monitor/changelog.md
- **Full Actor documentation**: https://apify.com/rookdev/public-tenders-monitor.md

## Changelog

### Unreleased (README and tests only)

- Store title changed to lead with the plain job, not monitoring (2 Nov 2026; see docs/research/2026-11-02-store-crowding.md). The Actor name in its URL and all behaviour are unchanged.
- Test fixture for the change-notice lookup replaced with a real capture (2 Nov 2026); the live drift check confirmed the lookup works as coded: no code change.
- FAQ explains the catch-up after a big first window: held-back notices arrive up to 1,000 a run, still as `new` (independent review, 31 Oct 2026, finding 8).

### 0.1.3 (2026-10-30)

Cost hardening from the cost-abuse audit (`docs/research/2026-10-30-cost-abuse-feeds.md`) and the template's new guards. The audit's five failing tenders scenarios are regression tests in `tests/test_cost_abuse.py`; same events and prices.

- **Cost guard** (template `CostGuard`, `MeteredBiller`, `RunContext`): a run stops starting new work once its compute and row writes are $0.02 ahead of 80% of what it has charged, checked before every request and retry wait and every 10 notices in the parse and filter loops. Sources not checked share one free `skipped` row. A run that earns nothing now costs at most about $0.024 at 1 GB, not $0.20 an hour until the timeout. `RUN_SUMMARY.cost` shows the estimate.
- **Spending limit:** a source starts only when the limit covers one check and one notice for it and for the sources already running. A $0.002 limit used to read every source in full and charge nothing; now it reads nothing.
- **Inputs:** at most 10 source lines per run (one free row for the rest, bad lines in one push); at most 100 keywords, exclude keywords, CPV codes and countries (a longer list is a free input row, never cut silently); each keyword list is one compiled pattern (1,001 exclusions cost 0.14 CPU-s per notice); 3 failed sources in a row end the run; error rows quote at most 5 problems.
- **Bounded scans:** TED is always asked for 250 notices per page (a cap of 20 used to mean pages of 20, up to 750 requests); a normal run stops a source after 10 pages in a row without a matching notice; monitors read Find a Tender and Contracts Finder from their last complete, fully delivered read minus one day instead of the whole window; every page is capped at 20 MB.
- **Open notices only in monitors:** monitoring runs neither send nor remember notices whose deadline has passed, and a remembered tender is forgotten when its deadline passes (notices without a deadline, such as awards, 60 days after they were last seen, as before), so the memory no longer grows with closed tenders. Normal runs still list closed notices.
- **CPU:** contact details are scrubbed out of descriptions only for the notices returned, not for every notice read (94% of a no-match scan's CPU).
- **Network:** TED's POST now goes through the template's fetch like every GET: 10 s to connect, 60 s per request, the private-address checks and the size cap; all requests share the run's per-host limiter (2 in flight per host, a host left alone after 10 failures in a row).
- **Rate limits (field test):** when Find a Tender or Contracts Finder answer `429` (or `503`) with a Retry-After longer than 30 s, as they did with `Retry-After: 120` when a run was started again a minute later, the source gets a free `skipped` row ("the service asked to wait 120 s; it will be read next run") instead of an `error` row, is not counted as a problem in the status message, and its memory is untouched.
- Failed, not-found and throttled sources are never charged (tested). PRICING.md has a new Cost protection section with the worst case per run; the README FAQ explains early stops, rate limits and the input limits.
- **24-hour cap per Apify account:** the cost guard's $0.02 per-run allowance now comes out of $0.20 of unpaid compute per account per 24 hours, across runs (template `AllowanceLedger`, one record in the fixed store `public-tenders-monitor-limits`), so runs started back to back cannot add up to a large unpaid bill. A run stopped by it says so in plain words in its free `skipped` row and status message ("this Apify account has used today's allowance...; it resets within 24 hours"). If the record cannot be read, the run goes ahead.

### 0.1.2 (2026-10-30)

- Terms audit: Open Government Licence v3.0 attribution and TED source acknowledgement added to the README and dataset views; a Retry-After longer than 30 seconds now ends that source for the run with a free row instead of an early retry.

### 0.1.1 (2026-10-30)

Fixes from the pre-publication code review (each has a regression test in `tests/test_review_regressions.py`):

- Monitoring memory records when each notice was last seen, and forgets a notice 60 days after it was last seen instead of 60 days after it was first seen. A long-running tender updated after two months is no longer sent and charged again as `new`. Memories saved by 0.1 fall back to the first-seen date.
- TED change notices for contract awards and planned tenders that name the changed notice by publication number (seen live) are matched to that notice: the number is resolved to its notice identifier from the pages already read, or with one batched TED search. One award and its correction are one notice, not two `new` rows. If the search fails, the source still succeeds and those change notices are tracked as before.
- Privacy: sentences in descriptions are dropped when they hold 9 or more digits once spaces, dots, dashes and brackets are removed (CPV codes, notice numbers, dates and amounts with a currency are set aside), a phone label followed by a number in more languages (`Tel.-Nr.`, `Telefonnummer`, `Telefoon`...), contact labels (`Kontakt:`, `Persona de contacto`, `Referente:`, `RUP`...), a capitalised "Contact Jane Smith" or a title before a name (`Herr`, `dott.`...). Sentences are no longer split after abbreviations such as `Tel.` or `dott.`. Normal descriptions with quantities, codes, lots and prices are kept whole (tested both ways).
- One out-of-range date (`0001-01-01T00:00:00+01:00`) no longer fails a whole source, and any record the parser cannot read is skipped and counted (`recordsSkipped` in the run summary, and in the status message) instead of failing the read.
- New rows go out oldest first, so when `maxNoticesPerSource` holds notices back, the ones closest to leaving the window go first; the status message says how many were held back. README and input schema now say what happens when more notices arrive per run than the cap.
- When a tender has several versions in one window, the newest (by publication date, then notice number, not the API's order) is chosen before filtering, so an older version with a superseded deadline or value is never shown when the newest fails a filter.
- Default memory raised to 512 MB for broad monitors with large memories. README and input schema recommend a window of 3 days or more for scheduled monitors.

### 0.1 (2026-10-29)

- First release: tenders, planned tenders and contract awards from EU TED (Search API v3), UK Find a Tender and UK Contracts Finder (OCDS), in one row shape with title, buyer, country, deadline, value, CPV codes, procedure and the official link.
- Filters: keywords (pushed into TED's full-text search), exclusions, CPV prefixes, buyer countries, minimum value in euros across currencies, notice type, publication window.
- Monitoring of new notices and of tenders whose deadline or value moved (TED change notices are matched to their procedure, UK updates to their OCDS process); nothing is ever reported removed, because each run reads a rolling window. Optional quiet first run.
- Find a Tender is read without its `stages` filter, which returns only notices under the old regulations, so Procurement Act 2023 notices are included.
- No personal data: contact points are never read, and sentences with e-mail addresses, phone numbers or named contacts are removed from descriptions.
- Safe billing from the Studio template: a source check is charged only after its rows are delivered; problem rows are free; an invalid filter stops the run before anything is fetched. Every row has a one-line `summary`, and the status message lists the first new tenders for Slack alerts.