Maintenance patch fixing an independent reviewer's release blocker in the state-v1 → v2 migration, plus the durable-state size budget and truncation/partial-failure reporting. No dataset-record change (schemaVersion stays 1.0.0) and no state-version change (the document stays version 2; the new legacyEligible field is additive). 171 tests pass; verified with lint, typecheck, build, the full test suite, and the fixture smoke:local run.
- P1 — a configuration created after the v1 → v2 migration can no longer inherit a pre-upgrade baseline. Adoption of a migrated legacy entry was gated on "this partition has a cursor", which becomes true for any configuration once it has run once. A configuration created after the upgrade was therefore ineligible on its first run (no cursor yet) and eligible on its second — at which point an unchanged pre-upgrade CVE it had never tracked itself was matched against a different configuration's v1 snapshot and silently suppressed as "unchanged", losing the new-record event and back-dating
firstSeen to a run that configuration never made. The exact set of pre-upgrade partition keys is now captured verbatim from the v1 document's cursors at migration time and persisted as legacyEligible. Only that frozen set may adopt; the grant is consumed the first time a partition actually adopts; and once no grant remains the legacy entries are dropped rather than left reachable. Truly pre-upgrade configurations still keep their cursors and firstSeen exactly as before. Adoption is now decided inside applyStateTracking from the state's own grant set, so no caller can widen it. A v2 document written before this field existed (Actor 1.0.5) grants nothing — adoption is a one-shot upgrade step it already had; at most it re-emits, once, a CVE its configuration had not re-observed since the migration.
- P1 — concurrent-merge semantics for the grants. A consumed grant is a tombstone, so
mergeStates merges legacyEligible by intersection (every other field is still unioned / furthest-wins). Unioning would resurrect a grant an overlapping run had already spent — or, when the other side is a not-yet-migrated v1 document, re-grant it indefinitely. Intersection is the fail-safe direction: at worst a pre-upgrade partition loses an unspent grant and re-emits its baseline once with a fresh firstSeen (over-reporting, never a suppressed change).
- P2 —
MAX_STATE_ENTRIES is now derived from the Apify payload limit instead of an unverifiable 250,000. The whole state document is one key-value record and the Apify client serializes it pretty-printed into a request bounded by the platform's 9 MiB MAX_PAYLOAD_SIZE_BYTES; at a measured ~330 pretty-printed bytes per entry, the old ceiling projected to ~82 MiB — roughly 9× over the limit, so a sufficiently long-lived state document would have failed to persist. The cap is now a conservative 12,000 entries ≈ 3.8 MiB (under half the limit, >5 MiB of headroom), and a test fills a real document to the cap — worst-case records, 16-hex partition keys, one cursor per partition — measures the exact pretty-printed payload and fails if the headroom erodes. Rolling-prune semantics are now documented and tested: at the cap the least-recently-seen entries are dropped, and a dropped CVE that reappears is re-emitted once as new (fresh firstSeen, empty changedFields) rather than suppressed. A single run tracks at most ~6,000 CVEs (3,000 forward + 3,000 recheck), so the cap binds only across many runs or many configurations.
- P2 — recheck-coverage truncation no longer masks a completed forward drain. A partly-unchecked recheck overlap (a change-detection gap below the cursor, which can never advance or regress it) was folded into the same
truncated flag as a forward-drain gap, so it suppressed lastSuccessAt and made a healthy configuration look like it had never completed a drain. The two are now separate: forwardTruncated governs the cursor's lastSuccessAt / lastTruncatedAt, recheckTruncated is reported on its own, and truncated remains the aggregate of both. LAST_RUN_SUMMARY gained both flags (additive).
partialFailures now includes partially-failed sources, not only hard failures. A cveIds run in which some per-ID lookups failed reports status: "ok" with an error and delivers an incomplete answer, yet was invisible in partialFailures; any source carrying an error is now listed there (and the partial NVD error is logged in the run's closing line).
Release-blocker fixes from an adversarial review: incremental liveness, cveIds outage reporting, and durable-state memory. No dataset-record change (schemaVersion stays 1.0.0); the persisted state document moves to version 2 with an automatic, non-destructive migration. 164 tests pass; verified with lint, typecheck, build, the full test suite, and the fixture smoke:local run.
- P0 — a selective configuration can no longer stall: base-filter rejections now advance the scan watermark. The forward drain was built only from records that passed the base filters, so a run whose backlog prefix is rejected at the base stage (
cvssMin, vendor/product/cpe, date ranges) had nothing to carry the cursor and re-scanned the same prefix on every run — permanently, for a configuration whose matches all sit behind that prefix. The cursor now advances over every forward record that was fully fetched and definitively classified, base rejections (decided from the NVD record alone) included. Safety is unchanged and enforced by construction: the watermark may only cross the contiguous oldest-first prefix that precedes the first record which was not definitively handled (a match dropped by maxItems, or a candidate left un-enriched by the 3,000-record budget), it is still clamped to NVD's completedThrough so an un-fetched lower-dated record can never be stranded, and it is still held entirely when a filter-required enrichment failed. New adversarial tests cover a zero-match base filter, a rare match behind a rejected prefix, the contiguity invariant (rejections above a dropped match do not advance), repeated multi-run progress through a 300-record backlog with no skips or duplicates, and page-budget truncation with rejections.
- P1 — a total
cveIds lookup failure is now reported as an NVD failure. If every per-ID lookup failed, fetchNvd returned status: 'ok' with zero records, which the run logged as a clean empty run: a full NVD outage was indistinguishable from "none of these CVEs exist". It now returns status: 'failed', so the run hard-fails, writes no state, and surfaces the error in LAST_RUN_SUMMARY.sources / partialFailures. Partial success is unchanged: if at least one lookup succeeded, those records are emitted and the cursor is held (completedThrough: null); a lookup that legitimately returns zero rows is still a clean, committed run.
- P1 — durable change-tracking state can no longer approach ~1 GB. Each entry stored a verbatim copy of every tracked field, including the full description and the joined reference list (~5 kB/entry, ~1.2 GB at the 250,000-entry ceiling). Entries now store a single positional vector of 12-hex-character per-field digests (~0.2 kB/entry, tens of MB at the ceiling — measured and asserted in a scale test) and no record content at all.
changedFields semantics, last-known-good enrichment carry-forward during an optional KEV/EPSS outage, firstSeen/lastSeen, merge/prune and concurrency behavior are all preserved; a structurally invalid entry is treated as absent rather than trusted.
- P2 — change-tracking entries are partitioned by configuration. Entries were keyed by bare
cveId, so a newly-created configuration's first run was silently suppressed as "unchanged" by a different configuration's history. Entries are now keyed by <partition>:<cveId>, matching the cursor's existing config isolation.
- State v1 → v2 migration. A version-1 document is migrated on read instead of being discarded: cursors carry over unchanged (a v1 watermark is always at or below where v2 semantics would put it, so no record can be skipped) and snapshots are reduced to digests under a reserved legacy partition, which the first post-upgrade run of an already-running configuration adopts — preserving
firstSeen and avoiding a spurious "everything changed" burst. A mode: full configuration (no cursor, re-emits its window every run anyway) has firstSeen re-stamped once.
- P2 — published/modified ranges are pushed into the NVD query. The
publishedFrom/publishedTo (or modifiedFrom/modifiedTo) pair that matches cursorField is now intersected into the fetched window, so a bounded historical range is queried directly rather than scanned and discarded — and completedThrough stops at the requested upper bound so the cursor is never advanced over a range that was deliberately not fetched. An empty effective window issues no NVD request and says so; the non-aligned pair (which NVD cannot combine in one query) stays client-side and now emits a warning naming it.
- P2 — a truncated recheck window is flagged. When the enrichment budget or the
maxItems cap left part of the recheckWindowDays overlap unchecked, the run reported truncated: false despite a real change-detection coverage gap. It is now reported as truncated.
LAST_RUN_SUMMARY gained scanned (forward records definitively classified this run) and cursorAdvancedTo (the committed watermark, or null when held) for observability of the above. Additive only.
Documentation-accuracy fix for the Cost & pricing section (no Actor behavior, input, or output change; schemaVersion stays 1.0.0). Verified with lint, typecheck, build, full test suite, and the fixture smoke:local run.
- README no longer misstates the default cost of a run. The remote
defaultRunOptions.memoryMbytes was 4096 MB while the README assumed a 1 GB default, understating observed CU consumption by 4×. The section now states the CU formula explicitly (CU = configured memory GB × runtime hours), records that the intended remote default is 1024 MB (set at deployment) and that memory is an overridable default rather than a limit, and notes that the CU column must be multiplied by memory_GB for any other setting.
- Removed the universal "$0.4 / CU" price claim. The CU rate is plan- and account-specific (observed rates include $0.20/CU), so the worked estimates now lead with CU, give the conversion
cost ≈ CU × your plan's $ / CU, and present costs as an illustrative range across the published spread instead of a single false quote.
- Added run memory to the "how to keep it cheap" levers: the Actor runs with
maxConcurrency = 1 and is bound by NVD rate-limiting, so memory above the 1024 MB default raises CU without shortening runs.
Filter-correctness fix (no output-schema change; schemaVersion stays 1.0.0). 142 tests pass; verified with lint, typecheck, build, full test suite, and the fixture smoke:local run.
- Bare-date
publishedTo / modifiedTo upper bounds are inclusive through the end of the boundary day again. These filters are documented as inclusive calendar-day upper bounds, but input normalization canonicalized a bare YYYY-MM-DD to midnight UTC, so a record timestamped later that same day was silently dropped. A bare upper bound now anchors to end-of-day (T23:59:59.999Z); an explicit ISO timestamp upper bound stays exact. Added filter-level and run/input-normalization regression tests covering a record later on the boundary day (retained) and one just after it (dropped).
Release-blocker hardening from an adversarial review (no output-schema change; schemaVersion stays 1.0.0). 94 tests pass; verified with fixture and live NVD + CISA KEV + FIRST EPSS smoke runs.
- Enrichment-filtered backlogs never stall at the 3,000-record enrichment budget. The cursor now advances on a scan/progress watermark (every candidate in the fully-enriched prefix is definitively classified) rather than only on emitted matches, so a
kevOnly/epssMin backlog whose first 3,000 records contain no matches still makes forward progress and later runs reach matches beyond the budget. The cursor is still held below any record that was not fully processed (page-budget/mid-window NVD truncation, or a filter-required enrichment outage).
- EPSS completeness is proven, not assumed. The effective batch size is hard-capped at 100 regardless of any injected value, an explicit
limit is sent, and each response is validated against the live FIRST contract (data array present, total ≤ returned rows, server limit ≥ batch size, offset == 0, no unrequested CVEs). A truncated/malformed batch fails the whole enrichment closed; a requested-but-absent CVE with total == returned is treated as genuinely unscored.
- Malformed "successful" source payloads can no longer become false-negative intelligence. NVD (
{} / missing vulnerabilities / inconsistent totalResults), KEV ({} / empty catalog / count mismatch), and EPSS (missing data) responses are rejected instead of being read as a drained empty window / "nothing exploited" / "no scores". A legitimately empty NVD window (totalResults: 0) is still accepted.
- Optional KEV/EPSS outages preserve the last known-good enrichment. Change-tracking carries prior KEV/EPSS/risk snapshot values forward during an optional outage, so an outage (and its recovery) no longer fabricates
changedFields or overwrites good state with failure placeholders; only genuine post-recovery changes surface. Documented the bounded recheck/full-run path for CVEs first seen during an outage.
- NVD pagination advances by the rows actually returned, not the requested page size, so a server-side page cap or a short non-final page can never leave a gap; an empty page below
totalResults stops safely (no infinite loop) and holds the cursor.
- P2/P3: recursive NVD child-node CPE extraction with vendor/product derived from the same capped CPE list (evidence consistency); invalid numeric inputs are rejected rather than silently clamped; README corrected on risk weighting (CVSS 60 / EPSS-probability 25 / KEV 12–15; EPSS percentile drives the bucket, not the score), byte-identical/idempotency claims (run-stamped fields + append-only dataset),
maxConcurrency = 1 operational contract, and concrete pricing/cost estimates + explicit FIRST EPSS attribution text.
Correctness and robustness fixes (no output-schema change; schemaVersion stays 1.0.0).
- EPSS batch truncation fixed. The FIRST EPSS API caps responses at 100 rows unless an explicit
limit is sent; the previous batch size of 120 silently dropped scores past the first 100 per batch, which could make genuinely scored CVEs look unscored and corrupt an epssMin / epssPercentileMin filter. Batches are now ≤100 IDs and send limit=<batch length>, so every requested score is returned (verified live enriching 3000 IDs across 30 batches).
- Page-budget cursor skip fixed. NVD does not return date-window pages sorted by the cursor field, so advancing the keyset cursor after a page-budget (or mid-window-failure) truncation could strand a lower-dated CVE on an un-fetched page below the cursor and skip it permanently.
fetchNvd now reports completedThrough (the end of the last fully-drained window) and the cursor is clamped to it; a cveIds lookup partial failure now holds the cursor entirely. Truncated records are still emitted and simply suppressed as unchanged once the cursor catches up.
cvssMin now accepts decimals (e.g. 7.5, 9.8) instead of integers only, matching how CVSS base scores are expressed.
- Added adversarial regression tests: same-timestamp keyset drain, page-budget cursor clamp, unchanged-suppression + upstream edit via recheck, summary/OUTPUT write-failure ordering, run-level config isolation,
cveIds partial-failure hold, epssPercentileMin block, partial NVD page failure, and EPSS batch/limit behavior. 72 tests pass.
Initial public release.
- Joins official NVD CVE API 2.0, CISA KEV catalog, and FIRST EPSS into one deduplicated, risk-prioritized CVE record (
recordId == cveId).
- Deterministic incremental cursor per (source × query configuration): advances only after successful dataset writes, never past failed/truncated/filtered regions.
recheckWindowDays re-surfaces upstream edits to already-committed CVEs without regressing the cursor.
- Documented order-independent dedup with explicit total-order tie-break.
risk.priority = critical / high / medium / low / unknown synthesised from CVSS + EPSS + KEV (KEV dominates; ransomware bumps to critical; documented heuristic, not an official standard).
- Per-source and per-sub-dataset failure isolation: NVD hard-fail, KEV outage, EPSS outage all hold the cursor and surface as
failed / truncated in the run summary.
- HTTP layer: retry/backoff with both delta-seconds and HTTP-date
Retry-After, clamped to a sane ceiling; rate-limit spacing for NVD (~6 s unkeyed, ~0.6 s keyed).
- 60 unit + adversarial + smoke tests pass; offline
smoke:local exercises the full pipeline against fixtures.
- Multi-stage
npm ci Docker build for byte-identical dependency trees.
- Attribution and source terms linked in the README; data is © the respective authorities.