# Changelog of EU & UK Public Tenders Scraper (TED + Find a Tender) (`atesen-software/eu-uk-public-tenders-monitor`) Actor

- **URL**: https://apify.com/atesen-software/eu-uk-public-tenders-monitor/changelog.md
- **Full Actor documentation**: https://apify.com/atesen-software/eu-uk-public-tenders-monitor.md

## Changelog

### Cross review, 7 Nov 2026 (actor version 0.2, build 0.2.5 and README-only 0.2.6)

- **Daily platform test vs. the Find a Tender rate limit.** The UK API answered HTTP 429 with `Retry-After: 120` on the very first request of the real cloud pre-fill runs, and the old client could wait 5 x 120 s for one request, far past the platform's 5-minute test limit. New input `maxRuntimeSeconds` (0 = no limit; console pre-fill 180). The HTTP client no longer starts a retry wait that would end after the run deadline (`maxRuntimeSeconds` or the platform timeout minus 45 s) and no request may outlive it. The source then ends as `time_limit` and keeps the notices it already read (TED rows, Find a Tender pages already scanned); the run is a success when at least one source delivered data. If every source failed or was stopped before delivering anything, the run fails instead of passing empty. The exit message names every source that did not finish normally ("Partial result: ...").
- TED stopped by the run deadline now reports `time_limit` instead of `ok`.
- Find a Tender / Contracts Finder pagination links must stay on the API's own https host.
- Personal data: UK numbers written as `(020) 7946 0958`, `020-7946-0958` or `020.7946.0958` are masked too.
- Monitor state cap lowered from 300,000 to 150,000 procedures per key (a full 300,000-entry state needed about 480 MB of RAM to load, too much for the 512 MB default).
- README: `maxRuntimeSeconds`, what deleting the monitor state does, the real Find a Tender rate-limit behaviour, the Find a Tender / Contracts Finder overlap, no claim about a platform start event. Store description: no longer claims "No contact persons" (names inside free text are not detected).
- Tests: 82 (was 67), including fake-429 runs on a virtual clock (`test/time-limit.test.js`).

### Publication prep, 7 Nov 2026 (actor version 0.2, build 0.2.4)

- Platform source (build 0.2.1) pulled into this folder. The binary test fixtures on the platform had been stored as text and were unreadable, so they were rebuilt from the official APIs with `scripts/make-fixtures.mjs` (window 23-26 Oct 2026, same notices as before). `npm test`: 67/67.
- Input schema: console pre-fill is now keywords `software`, CPV `72000000`, sources TED + Find a Tender, 3 days (fewer Find a Tender pages, which the UK API rate-limits), 5 notices per source, so the daily platform test finishes in seconds and stays cheap.
- Default run memory 512 MB (was 4096 MB on the platform), timeout 30 min.
- README rewritten for buyers (EU tenders API, TED notices scraper, UK Find a Tender, public procurement monitor, CPV filter), fixed price removed from README and Store text, FAQ and source terms added.
- `.apifyignore` keeps research, samples, tests and scripts off the platform (and keeps binary fixtures from being corrupted by upload).

### 1.1.1 (actor version 0.2), 26 Oct 2026, unreleased

Fixes from the QA review of 1.1.0 (`grok-qa/tenders/QA.md`, round 3).

#### Fixed

- **Status when the scan safety limit is hit.** If TED stops at the monitor scan safety limit, the source's status row now says `scan_limit` instead of `ok`, and its message says the results may be incomplete. The existing page caps (`TED_MAX_PAGES`, `OCDS_MAX_PAGES`) now give `page_limit`. Budget and time limits still take precedence (`charge_limit_reached`, `time_limit`). The values are exported from `src/run.js` as `SOURCE_STATUSES`.
- **Blocked rows no longer count toward `maxResultsPerSource`.** Notices withheld by the personal-data guard used to count toward the TED limit. Now only notices that are actually output count, in every source and in both modes. The rows not counted are unchanged, duplicate and blocked ones. Blocked rows are counted in the source summary (`blocked`), and the status row message mentions them.
  - `fetchTed`: the option `countUnchanged` was replaced by `monitor`. `'uncounted'` is now honoured in all modes; the scan safety limit and 100-notice pages still apply only in monitor mode.
  - Side effect: outside monitor mode, a duplicate TED row (in practice never seen) no longer uses up the limit either.
- **`anywhere_in_country` notices now have a country.** TED's `anyw-cou` code usually has no country code with it, which left `performanceCountries` empty. When the scope is `anywhere_in_country` and no country is given, `performanceCountries` is now `[buyerCountry]`. A country that TED does give wins. Other scopes are not filled. (`tedPlaces(values, buyerCountry2)` in `src/place.js`.)

#### Documentation

- README, monitor tips: the "second run returns 0 notices" example now states that this holds only because run 1 covered the whole window. If run 1 stopped at `maxResultsPerSource`, the next runs return (and charge for) the older notices still in the window until the backlog is cleared. A new tip explains this and how to avoid it.
- README, Output: a list of status row values (`ok`, `scan_limit`, `page_limit`, `charge_limit_reached`, `time_limit`, `error`, `input_warning`, `invalid_input`).
- README, Place of performance: the buyer-country fallback for `anywhere_in_country`.
- Dataset schema: `status` now has an enum. It covers notice statuses (open, closed, awarded, planned, updated, cancelled, withdrawn, unknown), status-row values and null, with a description. The `performanceCountries` description mentions the fallback.
- Input schema: the `maxResultsPerSource` description mentions `scan_limit` and the backlog behaviour.

#### Tests

- New `test/v111.test.js` with 5 tests:
  - `scan_limit` status through `runSources`: an endless TED feed where every notice is unchanged.
  - A window exhausted normally still reports `ok`.
  - The schema `status` enum covers every status-row value and every fixture notice status.
  - 3 PII-poisoned TED notices are blocked and do not use up a limit of 5: 5 clean notices are still output.
  - `anywhere_in_country` buyer-country fallback, in `tedPlaces` and `normalizeTed`.

- `test/v11.test.js`: the scan-limit test now passes `{ monitor: true }`.

- Total: 67 tests (was 62), all passing.

- Versions: `package.json` / `package-lock.json` 1.1.1. `.actor/actor.json` stays 0.2, because Apify versions are major.minor only and 0.2 has not been published; the next build of 0.2 picks this up.

### 1.1.0 (actor version 0.2), 26 Oct 2026, unreleased

This release is based on the frozen 1.0.0 build (QA V9, `tenders-actor/`). It covers the three non-blocking notes from QA (`grok-qa/tenders/QA.md`, round 2, "Kalan küçük notlar").

#### Added

- **Structured place of performance.** Every row from every source now has four new fields:
  - `performanceCountries`: ISO 3166-1 alpha-2 codes. They come from TED ISO-3 codes, NUTS prefixes (`EL` → `GR`, `UK` → `GB`) and country names.
  - `performanceNutsCodes`: NUTS level 0–3 codes, such as `DE111` or `UKM82`.
  - `performanceRegions`: free-text region names that are not codes. Contracts Finder gives these, e.g. "London" or "Rest of the World".
  - `performanceScope`: `anywhere`, `anywhere_eea` or `anywhere_in_country`, mapped from TED's `anyw`, `anyw-eea` and `anyw-cou` codes.
  - The new logic is in `src/place.js`.
  - The legacy `placeOfPerformance` field is **unchanged**: raw codes as published, cut at 20. It is kept for backward compatibility, and its dataset-schema description now points to the new fields.
  - The new fields hold up to 100 codes. This matters for notices with many NUTS codes, where the 20-entry cut on the legacy field used to drop the country code (e.g. TED 733660-2026 has 59 NUTS codes).
- **README example for acronyms and short keywords.** Type acronyms in capitals (`IT`, not `it`). Keywords of 3 characters or fewer match whole words only. Combine keywords with CPV codes for full coverage. A tip was also added to the `keywords` description in the input schema.

#### Changed

- **Monitor mode: `maxResultsPerSource` now counts only notices that are output (new or changed).**
  - Unchanged notices are skipped and no longer use up the limit. Before this, a small limit could stop a TED run after N unchanged notices and miss newer changes further down.
  - TED (`src/ted.js` `fetchTed`): the `onRow` callback can return `'uncounted'`. Stats now include `emitted` and `uncounted`.
  - Monitor runs always request 100-notice pages.
  - New safety limit `monitorScanLimit = max(1000, 20 × maxResultsPerSource)` scanned notices. When it is reached, the run sets `stoppedBy: "scanLimit"` and logs a warning. The existing 100-page cap (`TED_MAX_PAGES`) also logs a warning now.
  - FTS / Contracts Finder (`src/run.js`, `src/ocds.js`): in monitor mode, every matching row in the window is evaluated (they are already in memory, so there are no extra requests). The cap is applied to emitted rows.
  - Without monitor mode, nothing changes: every returned notice counts, and small runs still use small TED pages.
- Versions bumped: `package.json` 1.1.0, `.actor/actor.json` 0.2.

#### Tests

- New `test/v11.test.js` with 7 tests:
  - Place-code classification.
  - TED and OCDS place normalization on the fixtures.
  - TED monitor cap: run 2 returns the next 5 new notices instead of 0, and repeated runs drain all 20 fixture notices.
  - The non-monitor cap is unchanged.
  - The scan-limit safety stop.
  - The FTS monitor cap.
- `npm test`: 62/62 (was 55/55). Against the frozen 1.0.0 code, 6 of the 7 new tests fail. The one that passes is the deliberate no-regression check for non-monitor runs.
- The dataset schema test now also covers the new fields. 81 fixture rows pass ajv validation against `dataset_schema.json`. `apify validate-schema` reports input, dataset and output schemas valid.
- Live local run without a token (`samples/run_v11.sh`): CPV 72000000, 2 days, TED + FTS, `maxResultsPerSource` 5, monitor mode.
  - Run 1: 9 new (TED 5, FTS 4).
  - Run 2: TED returned **5 more new notices** with 5 unchanged skipped; FTS returned 0 new with 4 unchanged.
  - With 1.0.0, run 2 would have returned 0 TED notices.