Monitor new federal contract and grant awards from USAspending.gov (US Treasury / DATA Act). Filter by agency, award type (contracts, grants, loans, direct payments), NAICS, PSC, recipient and amount; one normalized record per prime award as it lands, with a new-since-last-run mode. Keyless, CC0.
2026-07-11 — QA fix: bound the prefilled/demo run (maxResults prefill 100)
Apify automated QA flagged the live Actor "Under maintenance" — with the prefilled input it
didn't finish inside the 5-minute test window. Root cause is volume, not a bug: the API is healthy
(probe = 200, 100 results/page, hasNext=true), but federal prime awards run to tens of thousands
per default 7-day window, and maxResults had no prefill (default 0 = unbounded), so QA
walked to the maxPages=100 guard = 10,000 records + one charge + one push each → over 5 min (and
billing the test account 10k events). Highest-volume source on the shelf; the low-volume siblings
pass QA on the same unbounded default.
Fix: add "prefill": 100 (one page) to maxResults in .actor/input_schema.json so a first/demo/QA
run is bounded and fast. default stays 0 — real users who want the full scheduled pull still set 0.
Verified: bounded fetch stops at 1 page / 100 records in ~1.5s; 55/55 unit tests still green.
0.1 — 2026-07-10 — slice 4: test/node --test suite (mocked HTTP, no network)
Added the formal offline unit suite (node --test, node:test + node:assert/strict, no new
deps), mirroring the openfda-enforcement/test/ layout. 55 tests, all pass; zero network.
test/usaspending.test.js — pure API-shape functions: buildSearchBody (empty→A/B/C/D + default
fields + no time_period; caller codes override + blanks dropped; time_period on ≥1 bound;
unknown dateType→new_awards_only; malformed bound ignored; sub-filter → nested-key mapping +
omit-when-empty; minAmount emit only when >0; limit clamp 1–100 incl. 0→100; page clamp ≥1;
fields empty→DEFAULT_FIELDS), toAmount (number/string/$/comma coercion, junk/∞/NaN→null),
toIsoDate (keep YYYY-MM-DD, trim ISO datetime, reject bad month/day/format), normalizeRecords
(fold to documented shape, coerce amount+dates, generated_internal_id→internal_id fallback,
unmapped requested fields pass through under raw key, skip non-object rows / non-array results).
test/fetch.test.js — HTTP layer with injected fetchImpl/sleep/post (no real clock/network):
backoffDelayMs/isRetryableStatus, postJson (first-success no-retry + POST body/headers assert;
null body→{}; dispatcher passthrough; 5xx & 429 retry-then-succeed with backoff schedule; no
retry on 400 carrying .status/.body; transport-error retry-then-exhaust; persistent-5xx give-up;
backoff cap), fetchAllPages (walk until hasNext=false, order-preserving; body clone/no-mutation;
maxResults early stop; maxPages guard; honour starting page; tolerate missing results/metadata;
require a body object).
test/filters.test.js — every pure helper: dedupeByAwardId (keep-first + generated_internal_id
fallback + keyless-never-collapsed + non-array), filterByAgency/filterByRecipient (ci/space,
empty=all, null-drop-when-active, non-array), filterByMinAmount (boundary-inclusive, ≤0/non-numeric
passthrough, null/non-finite drop when active), filterNewOnly (unseen+tag+advance, fallback key,
keyless kept-not-tracked, second-run suppression), applyFieldProjection (narrow but keep
isNew/fetchedAt, empty=passthrough same-ref, absent key not fabricated), applyMaxResults.
Implemented the pure filter helpers in filters.js (order-preserving, null-tolerant, no I/O):
dedupeByAwardId (keep-first on award_id→generated_internal_id; keyless records kept),
filterByAgency/filterByRecipient (case-insensitive on awarding_agency/recipient_name;
empty = all; null-field records dropped while a filter is active), filterByMinAmount
(drop award_amount < min; 0 = no min; null amount dropped when a min is set),
filterNewOnly(records, seenSet) (return only unseen ids, tag isNew:true, advance the KV
watermark), applyFieldProjection (empty = passthrough; always preserves isNew/fetchedAt),
applyMaxResults (0 = no cap).
Implemented the main.js Actor runtime replacing the scaffold stub: read input → derive a bounded
time window (since→trailing 7-day default; until→today; always sent, since
spending_by_award expects a time_period) → buildSearchBody (empty awardTypeCodes→A/B/C/D;
sub-filters mapped) → fetchAllPages (postJson w/ backoff; optional undici proxy dispatcher;
page until hasNext=false/maxResults) → normalizeRecords → dedupeByAwardId →
filterByAgency/filterByRecipient/filterByMinAmount → filterNewOnly (when monitorNewOnly,
from KV SEEN) → applyFieldProjection (passthrough; field narrowing is server-side) →
applyMaxResults → push one dataset item per award, charging one award-returned PPE event
each. _noData/_error control records carry the filters summary and are NOT charged. KV SEEN
watermark persisted only after a fully successful, uncapped pass (charge-limit guard).
PPE value stays owner-deferred — the award-returned event is emitted in code; intended price
lands in docs/PRICING.md (slice 6). No on-platform pricing set.
Verified offline (no network): 20-check filters.js smoke (dedupe/keyless/fallback, agency+recipient
case-insensitivity, min-amount boundary+null-drop, new-only tag+set-advance+fallback key, projection
passthrough+metadata-preservation, cap) + 17-check end-to-end pipeline smoke with a mocked post
(empty→A/B/C/D, time_period+sub-filter mapping, 2-page walk, cross-page dedupe, agency/min/new-only
chain, maxResults page-walk early stop). All src/*.jsnode --check clean. Formal node --test
suite is slice 4.
Implemented the API layer (network + raw-shape module; filters.js/main.js never touch the API):
buildSearchBody({awardTypeCodes, since, until, dateType, filters, fields, page, limit, sort, order})
→ the POST /search/spending_by_award/ JSON body: filters.award_type_codes (empty → prime
contracts A/B/C/D), time_period:[{start_date,end_date,date_type}] (emitted when a bound is
present; unknown dateType → new_awards_only), and the sub-filters agencies
([{type:"awarding",tier:"toptier",name}]), naics_codes, psc_codes, recipient_search_text,
award_amounts:[{lower_bound}] (each omitted when empty); fields (empty → DEFAULT_FIELDS),
page≥1, limit clamped 1–100, sort/order.
postJson(url, body, {fetchImpl, sleep, onRetry, retries, …}) — injectable POST with
Content-Type: application/json + descriptive User-Agent, deterministic exponential backoff on
5xx/429/transport errors, no retry on other 4xx, carries .status/.body.
fetchAllPages({url, body, maxPages, maxResults, post, …deps}) — pages until
page_metadata.hasNext===false (or maxPages/maxResults guard), clones body per page (no
mutation), returns {results, pages, pageMetadata}; dedupe left to filters.js.
normalizeRecords(json, {awardTypeCodes}) — folds results[] into the documented record shape
(defensive missing→null, numeric award_amount, YYYY-MM-DD dates, carries
generated_internal_id; extra caller-requested fields pass through under their raw key).
Verified offline (no network) via an ad-hoc smoke harness: body construction + defaults + sub-filter
mapping, limit/page clamps, dateType fallback, postJson retry-then-succeed on 500 and
no-retry on 400 (with .status/.body), fetchAllPages pagination walk + maxResults early stop,
and normalizeRecords folding/null-handling/extras-passthrough. All src/*.jsnode --check clean.
Formal node --test suite is slice 4.
0.1 — 2026-07-10 — slice 1: scaffold
Scaffolded from the openfda-enforcement / usgs-water-monitor reference implementations
(structure, patterns, PPE model).
Fetch-based Apify Actor (no browser): apify@^3.2.6 only, apify/actor-node:20 base, 256–512 MB.
