TED Tenders Scraper (EU & US Public Procurement Data) avatar

TED Tenders Scraper (EU & US Public Procurement Data)

Pricing

from $3.20 / 1,000 tender delivereds

Go to Apify Store
TED Tenders Scraper (EU & US Public Procurement Data)

TED Tenders Scraper (EU & US Public Procurement Data)

Extract official EU procurement notices from TED and U.S. federal contract awards from USAspending as unified delta events. Supports saved queries and source filtering. Public data, pay-per-result, schema-validated.

Pricing

from $3.20 / 1,000 tender delivereds

Rating

0.0

(0)

Developer

Steadfetch

Steadfetch

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

5 days ago

Last modified

Categories

Features

Limited permissions

Share

Extract official EU procurement notices from TED and U.S. federal contract awards from USAspending as unified delta events. Each run delivers only new, previously unseen contract awards or notices, ensuring zero duplicate billing.

Tested daily. Last verified: 2026-10-30

Saved searches, a source filter, one unified schema and delta-only results: each run returns only the notices and awards it has never delivered before.

What it does

  • Calls two official, keyless JSON APIs:
    • TED Search API v3 (POST https://api.ted.europa.eu/v3/notices/search);
    • USAspending (POST https://api.usaspending.gov/api/v2/search/spending_by_award/).
  • Runs 1-50 saved queries per run. TED queries accept text, cpv, countries and minValueEur; USAspending queries accept text, agencies and naics. A query with only free text (or no filters) runs on both sources.
  • Returns one row per notice/award in a unified camelCase schema, with a mandatory source field (ted or usaspending).
  • Delta / dedupe: delivered (source, noticeId) pairs are stored in the named key-value store ted-tender-monitor-state. A second run with the same filters returns 0 new rows and therefore 0 result charges.
  • Enforces TED's fair-use limits locally: 700 requests/minute and 600 downloads/6 minutes, with exponential backoff on 429/5xx.
  • TED rows always carry a tendering deadline. Each row's deadlineAt is the published submission deadline, or the request-to-participations deadline for restricted and negotiated procedures. Award, planning and modification notices have no deadline, so they are never delivered from TED (a null deadlineAt means the row comes from USAspending).
  • No person-level data. TED contact/person/UBO fields are never requested and are scrubbed again before delivery (GDPR). USAspending rows use contract and organisation fields only.
  • Discards schema-invalid items instead of failing the whole run; discarded items are never pushed and never billed.

How to use

  1. Open the Actor on the Apify Store and press Start, or run it from the CLI:

    $apify call steadfetch/ted-tender-monitor --input='{"source":"all","queries":[{"text":"software"}],"maxResults":10,"monitorMode":false}'
  2. Add one to 50 saved queries. Free-text-only queries run on both sources; CPV/country/EUR filters target TED and agency/NAICS filters target USAspending.

  3. Read, export or stream the default dataset as JSON or CSV (the Store table view works too).

  4. For monitoring, schedule the run daily with the same queries: with monitorMode on (the default), each run returns only rows it has never delivered before, and a run with 0 new rows costs 0 result charges. Turn monitorMode off (the Store form prefills false) for a deterministic backfill that re-emits matches even after they were delivered once.

Input

The Store example run (exampleRunInput):

{
"source": "all",
"queries": [
{ "text": "software" }
],
"maxResults": 10,
"monitorMode": false
}

All filters in one run:

{
"source": "all",
"queries": [
{ "cpv": ["48000000"], "countries": ["ES"], "minValueEur": 100000, "text": "software" },
{ "agencies": ["Department of Energy"], "naics": "511210", "text": "cloud" }
],
"maxResults": 100,
"monitorMode": true
}
FieldTypeDefaultDescription
sourceted, usaspending or allallWhich source(s) to query.
queriesarray, 1-50requiredSaved searches. All filters inside a query are optional.
queries[].textstring-Free-text keywords, applied to TED and/or USAspending.
queries[].cpvarray of strings-TED CPV code or prefix (48000000, 48).
queries[].countriesarray of strings-TED buyer countries, ISO2 or ISO3.
queries[].minValueEurnumber-TED only: minimum published estimated value in EUR. Rows without a published EUR value are excluded.
queries[].agenciesarray of strings-USAspending top-tier agency name, slug or abbreviation (Department of Energy, DOE).
queries[].naicsstring-USAspending NAICS code.
maxResultsinteger, 1-10000100 (Store form prefills 10)Maximum new rows delivered in the run, across all queries and sources.
monitorModebooleantrue (Store form prefills false)When on, rows already delivered by a previous run are not pushed or charged. Turn it off for a deterministic backfill.

Each run looks back 7 days on both sources (TED publication date, USAspending last modification date), so a monitor never walks the full historical corpus. Duplicates across queries are removed within the run.

Example output

Real item from the saved USAspending fixture (test/fixtures/usaspending-awards.json):

{
"source": "usaspending",
"noticeId": "CONT_AWD_36C25024F0188_3600_47QTCA19D00MV_4732",
"title": "GEO TAB - SAGINAW",
"buyer": "Department of Veterans Affairs",
"country": "US",
"cpv": [],
"value": 13162,
"currency": "USD",
"publishedAt": "2023-12-01",
"deadlineAt": null,
"periodEndAt": "2024-11-30",
"url": "https://www.usaspending.gov/award/CONT_AWD_36C25024F0188_3600_47QTCA19D00MV_4732",
"firstSeenAt": "2026-10-19T12:00:00.000Z",
"scrapedAt": "2026-10-19T12:00:00.000Z"
}

TED rows use the same fields:

FieldTEDUSAspending
noticeIdTED publication numbergenerated_internal_id
titleNotice title (English preferred)Award description, or award ID and recipient
buyerContracting authorityAwarding agency
countryBuyer country, ISO2US
cpvCPV codesEmpty array (the source uses NAICS)
value / currencySummed lot estimate and its currencyAward amount and USD
publishedAtPublication dateAward start date
deadlineAtTendering deadline (submission or request-to-participations stage)null
periodEndAtnullEnd of the period of performance
firstSeenAtWhen this Actor first delivered the rowSame

value and currency are exactly what the source publishes: no conversion and no invented amounts. url points to the official notice/award page.

Pricing

  • 4.00 USD per 1,000 new rows (apify-default-dataset-item = 0.004 USD per row), plus the Apify-recommended apify-actor-start event (0.00005 USD per run, first 5 seconds of compute included).
  • monitor-check: 1.00 USD per 1,000 checks (0.001 USD per checked record), charged in monitorMode only, once per record actually checked. A check that finds no change still bills the check but no result event.
  • A re-run with no new rows emits 0 rows = 0 result charges.
  • Empty results, errors, retries and previously delivered rows are not charged.

Works on the Apify free plan - about 1,249 tenders for the $5 monthly credit ((5.00 USD - 0.0002 USD run-start at the default 4 GB memory) / 0.004 USD per tender).

Cost per 1,000 results

Measured locally on 22 October 2026 with npm run fixture:bench (saved official fixtures, no APIFY_TOKEN, 0 Apify credit):

  • extract 1,000 rows + validate them against the dataset schema: about 19 ms of local CPU;
  • at 256 MB and 0.20 USD per GB-hour, that is about 0.0000003 USD of compute per 1,000 rows;
  • both APIs are keyless and free, no proxy is used.

Measured cost per 1,000 results is therefore well below 0.001 USD; at 4.00 USD per 1,000 the gross margin is above 99.9%. Full method in docs/measurements.md.

Both sources were probed without credentials before building:

  • TED: POST https://api.ted.europa.eu/v3/notices/search returned HTTP 200 with 3 notices and totalNoticeCount: 1995 for a free-text + date query. No API key is required, so the bulk daily download fallback is not needed.
  • USAspending: POST https://api.usaspending.gov/api/v2/search/spending_by_award/ returned HTTP 200 with 2 awards. The API is public and keyless.

Terms-of-service sweep, both sources CLEAR:

  • TED legal notice: procurement notices in the Supplement to the Official Journal "can be freely reused, for commercial or non-commercial purposes"; editorial content is CC BY 4.0. No clause forbids automated extraction. https://ted.europa.eu/robots.txt only disallows login, private dashboards and internal Liferay patterns; the API host is not restricted (/robots.txt 404).
  • USAspending: U.S. federal government data in the public domain (17 U.S.C. section 105), documented public API with no login or key. The API host has no robots rules (/robots.txt 404); the www site robots file does not apply to the API host. No anti-scraping clause in the endpoint docs.

No login, paywall, CAPTCHA or private content is accessed.

Monitor state and backfill

  • Delivered pairs live in the named key-value store ted-tender-monitor-state (key seen-notices), with (source, noticeId) keys.
  • Entries are retained for 365 days and capped at 200,000 pairs, newest first.
  • To backfill from scratch, delete that named store in the Apify Console before the run. Backfilled rows are new to the state and are billed once.

Reliability

  • Both source APIs (TED and USAspending) are retried on transient failures (429/5xx, network errors, request timeouts, invalid JSON) with exponential backoff and capped retry-after sleeps. Every request carries a hard timeout, so a hung connection cannot exceed the run window.
  • If one source stays unavailable after retries (429/5xx, network errors, timeouts), the run still delivers what the other source provides and logs a warning; the persistent delta window picks the missed rows up on the next run. The run fails when the input is invalid, when a source rejects a request permanently (HTTP 4xx such as 400/401/403/404/422), or on any other unexpected error: a monitor that can no longer talk to a source must fail loudly instead of silently delivering an empty dataset.

Upgrading from v1

v2 is a breaking schema change, intentional and documented in the build: v1 emitted snake_case TED-only fields (notice_id, buyer_name, contract_value, cpv_codes, ...). v2 emits the unified camelCase schema above and adds source, firstSeenAt and periodEndAt. Update any consumer that reads v1 field names. The Actor version is now 0.2.

Other Steadfetch Actors

Part of a suite of web-data Actors that share one envelope convention and pay-per-event pricing:

FAQ

Does it need an API key? No. Both sources are public and keyless.

What if I start a run with no input? You get the latest 7 days of EU TED notices and US federal contract awards from both sources, capped at maxResults (100 by default). A run whose input body is not a JSON object (for example [], "x" or 123 sent to the run API directly) fails with an invalid-input error: send an object, {} is enough, to get the default query. Only other malformed input values (bad source, too many queries, wrong field types) fail a run.

What is the difference between TED and USAspending? TED publishes EU procurement notices with deadlines and CPV codes. USAspending publishes U.S. federal contract awards with a period of performance and USD amounts. One Actor normalises both.

Why is buyer the awarding agency for USAspending? In the unified schema buyer means the public body buying the goods or services. The award recipient appears in title (the description or "award ID - recipient").

Why is cpv empty for USAspending? USAspending classifies contracts with NAICS, not CPV. The Actor never copies NAICS into cpv.

Why did my second run return nothing? It is a monitor: already delivered (source, noticeId) pairs are not repeated or billed. Delete the named state store to backfill.

How do I run only one source? Set "source": "ted" or "source": "usaspending".

Support

Report a bug or a data question through the Actor's Apify Store page (Issues tab) or the Steadfetch Actors repository issues. Include the run ID, the exact input JSON and the noticeId of the affected record so it can be reproduced.

Development and tests

Everything below runs offline against saved official fixtures and spends no Apify credit:

npm install --include=dev
npm test # lint + typecheck + Vitest
npm run fixture:bench # local cost measurement, no APIFY_TOKEN
npm run publish:check # Steadfetch pre-publication gate
npm run build # tsc production build

The suite covers both extractors, schema acceptance and rejection, request construction, agency resolution, query-to-source routing, the two-source orchestration and the persistent delta (a second run with the same filters returns 0 new rows).

Maintenance

This Actor is tested daily (2026-10-29) by the Steadfetch catalog QA workflow with fixture-based tests and a minimal platform run (source: all, one saved query, 10 rows). Last recorded daily QA pass on the published Actor: 2026-10-22. In this repository, schema and fixture tests run in CI on every push. If a source changes a field, schema validation discards the invalid rows and logs them instead of delivering bad data.