Tender & Procurement Feed Normalizer

Normalize tender records into a clean, deadline-sorted dataset. Each clean row preserves its complete original source fields. The REPORT key-value record retains every rejected row, original input, and rejection reason. Possible duplicates are retained in the report rather than the clean dataset.

Provide records, or a public HTTPS sourceUrl serving UTF-8 CSV/JSON (an {"records": [...]} JSON wrapper is unwrapped). URLs must use port 443 and cannot redirect, carry credentials, or resolve to private addresses. Maximum download: 5 MiB; maximum input: 10,000 rows. XML is unsupported.

{"records":[{"title":"Road maintenance","deadline":"2027-01-15","value":"USD 180,000"}],"currency":"USD"}

Live Actor build is 0.2.2 (deployed 2026-09-21), running this source tree (normalizerVersion 0.2.0 in REPORT).

Dates

Use ISO dates to avoid day/month ambiguity. Supported: ISO YYYY-MM-DD with optional time, fractional seconds and offset (2027-01-15, 2027-01-15T12:30:45.123Z, ...+02:00); %d/%m/%Y, %m/%d/%Y, %d.%m.%Y, 15 January 2027. Ambiguous slash dates (01/02/2027) are rejected. Dates without times mean midnight UTC; expired compares against run time.

All-digit values follow an explicit grammar: exactly 8 digits is a compact YYYYMMDD date (20270115); up to 10 digits is unix seconds; exactly 13 digits is unix milliseconds. 11–12 digits is rejected as ambiguous rather than guessed.

Money

Amounts accept unsigned decimals with up to two decimal places, comma thousands grouping, or consistently grouped European notation. Negative values, ranges, malformed grouping, and ambiguous dot grouping are rejected. Missing values are allowed; missing titles or deadlines are reported.

Currencies come from ISO codes in the value (USD 100, 100USD), symbols ($, €, R$), or an explicit currency column (currency, or a fieldMap target). Codes are matched on letter boundaries so USDTOKENS is not read as USD; symbols match case-insensitively, so a lowercased r$ is still BRL. RMB is canonicalized to CNY so the two spellings agree. Symbols $ and ¥ assume USD/JPY — declare explicit codes for other dollar/yen currencies. A declared currency that conflicts with the value's own currency is rejected (currency_conflict); nothing is inferred and no conversion is performed. The optional currency input must be a three-letter ISO code and flags rows whose currency differs (currency_mismatch_expected).

Duplicates and conflicts

Two rows are the same notice only when normalized title, deadline, buyer and reference all match. A different buyer or reference always survives — two authorities can run identically titled tenders on the same day. An exact repeat is reported duplicate; the same identity with a different stated value or currency is reported conflict and never silently merged. A missing amount or currency is a wildcard: an enriched copy is a duplicate, not a conflict. Only earlier clean rows can suppress later copies, so an invalid row cannot hide its corrected copy.

Errors and REPORT

Every issue carries a human message plus a stable machine errorCode: bad_record, missing_title, bad_title, missing_deadline, bad_deadline, bad_value, bad_currency, currency_conflict, currency_mismatch_expected, duplicate, conflict, withheld_billing. REPORT includes normalizerVersion, row counts (rowsClean counts delivered rows; rowsPassed counts every row that passed validation, including billing-withheld ones), duplicates, conflicts, errorCounts (issue occurrences per code — one row can carry several codes, so counts can sum higher than rowsWithIssues), and every failed row with full provenance.

Dataset rows preserve sourceFields verbatim — including text a spreadsheet would interpret as a formula. Treat spreadsheet exports as untrusted input; the raw provenance is deliberate and is never silently sanitized.

fieldMap

Maps your column names to standard fields (title, deadline, value, buyer, reference, currency). An explicit mapping beats the built-in aliases for that field: if you map a column to reference, an id column no longer outranks it. Sources match case/separator-insensitively, exactly like the aliases. Mappings with unmatched sources or non-standard targets are inert — but mapping a column does not remove its alias role for other fields, so one column can feed two fields; check mappedFrom. mappedFrom records the source column per field.

Offline demo and tests

python scripts/run_tender_demo.py    # golden fixtures: messy CSV + fieldMap, and direct JSON
python scripts/verify_all.py         # unit suites plus demo, exits non-zero on failure
python -m unittest discover -q       # unit suites only

examples/tender_feed/ has the fixture pack (CSV, JSON, field map, golden outputs); examples/apify_input/ has paste-ready fictional Actor inputs. Everything is fictional and offline.

Use as an integration (dataset input)

Point this Actor at another Actor's output instead of pasting records: set datasetId, or trigger it from another Actor's Integrations tab — the platform then supplies payload.resource.defaultDatasetId automatically. Only Apify platform datasets are fetched (alphanumeric IDs, at most 10,000 rows, batched reads) — never arbitrary URLs or caller credentials. Example static input for the integration form:

{"datasetId": "{{resource.defaultDatasetId}}", "currency": "USD"}

This is the pipeline pattern: any tender/procurement scraper Actor produces raw rows; this Actor turns them into an auditable clean table.

Cost math (published pricing)

$0.00005 per run start plus $0.001 per delivered clean row; rejected rows are never charged. Examples: 100 clean rows = $0.10005; 1,000 clean rows = $1.00005. A run's maximum charge is bounded by its maxTotalChargeUsd; rows beyond the budget are withheld (never charged) and listed in REPORT with withheld_billing. Raw scraper Actors in the Store charge several times more per raw row; this Actor's job is to make those rows usable. Prices are configured in the Apify Console and may change there; check the Store page for the current values.

Operator billing hook (disabled by default)

Configure a positive-price pay-per-event event named normalized-tender, remove apify-default-dataset-item from that Actor's pricing, and require a positive finite run spending limit before setting NORMALIZER_ENABLE_BILLING=1. An optional platform start event is separate; its already-charged cost is included in the SDK's remaining budget. Each delivered clean row uses the SDK's charged push_data; rejected rows do not generate this custom event. Withheld rows retain only row index, raw source data, and reason in REPORT, not their normalized output. Missing/invalid pricing, non-PPE mode, duplicate synthetic dataset charging, invalid/unbounded spending limits, and mismatched charge acknowledgements fail the run. No prices or remote settings are configured by this source tree. See https://docs.apify.com/sdk/python/reference/class/Actor#push_data.

Billing writes are not a transaction: SDK 2.7.3 persists dataset rows before calling the charge API. A charge transport failure makes the run fail; do not automatically retry failed paid runs without reconciliation. Cloud owner-run checks verified clean-row event counts, rejected-row handling, and spending-limit enforcement. They do not demonstrate an external customer payment.

No scraping, revenue, accuracy, or publication approval guarantees. Actor pricing and platform charges depend on the published Console configuration.