---

📑 Government Tenders Scraper v3 — USASpending, TED, UK Contracts & Custom Portals

Extract public procurement opportunities, awarded contracts, and tender notices from official government sources worldwide.
Uses free, no‑auth APIs (USASpending.gov) and HTML scraping for TED Europa, UK Contracts Finder, and any custom procurement portal. Get structured data on tender ID, title, issuing authority, recipient, value, deadline, category, country, and more.

---

💡 What is the Government Tenders Scraper v3?

This actor collects real, live tender and contract data from multiple official government sources. Unlike generic scrapers that fail on JavaScript‑heavy sites (like SAM.gov), this version uses:

• USASpending.gov API – Free, no API key, returns awarded US federal contracts (real data, billions of dollars).
• TED Europa (Tenders Electronic Daily) – HTML scraping of EU public procurement notices.
• UK Contracts Finder – HTML scraping of UK government tenders.
• Custom URL support – Any government procurement portal; the actor uses JSON‑LD, HTML cards, or link scanning.

No authentication required. Residential proxies are supported but not required for most sources. Output is clean JSON, ready for business intelligence, bid tracking, or market research.

---

📦 What Data Can You Extract?

🧩 Data Type	📋 Description
🆔 Tender ID	Unique identifier (e.g., USA-019810652655).
📄 Title	Description of the procurement.
🏛️ Issuing Authority	Government agency (e.g., NASA, Dept of Veterans Affairs).
🏢 Recipient	Awarded company (for USASpending) or empty.
📅 Deadline	Closing date (YYYY‑MM‑DD, if available).
📆 Posted Date	Publication date (if available).
💰 Value	Contract value (e.g., $2,511,478,245).
🔗 Link	Direct URL to the tender or award page.
🏷️ Category	Auto‑detected from title: IT, Construction, Healthcare, etc.
🧾 NAICS Code	(US only) Industry code.
📍 Location	City/state (US) or country.
🌍 Country	United States, European Union, United Kingdom, etc.
📝 Description	Full text (truncated).
✅ Status	Open (for active tenders) or Awarded.
📡 Source	USASpending.gov, TED Europa, Contracts Finder UK, or domain name.
⏱️ Processed At	ISO timestamp.

---

⚙️ Key Features

• Multiple Official Sources – USASpending (US federal contracts), TED Europa (EU tenders), UK Contracts Finder, plus any custom URL.
• No API Keys – All sources are public and free.
• Residential Proxy Ready – Optional but useful for heavy scraping or when behind corporate firewalls.
• Smart Category Detection – Automatically classifies tenders into 10+ categories (IT, Construction, Energy, Healthcare, etc.).
• Bulk URL Support – Scrape multiple custom procurement portals in one run.
• Deduplication – Prevents duplicate entries across sources.
• Clean JSON Output – Consistent field names, nulls omitted, ready for Excel, SQL, or dashboards.

---

📥 Input Parameters

The actor accepts a JSON object with the following fields:

Parameter	Type	Required	Default	Description
sources	array of strings	No	["usaspending","ted_europa","uk_contracts"]	Which sources to scrape. Options: usaspending, sam_gov (alias), ted_europa, uk_contracts.
target_urls	array or string	No	–	List of custom procurement portal URLs. Can be newline‑separated string.
target_url	string	No	–	Single custom URL (alternative to target_urls).
keyword	string	No	–	Filter results by keyword in title/description.
category	string	No	–	Filter by category (e.g., IT / Technology).
country	string	No	–	Filter by country (e.g., United States, European Union).
max_results	integer	No	20	Maximum number of tenders to return.
proxyConfiguration	object	No	–	Apify proxy configuration (optional, residential recommended for large runs).

Example Input (USASpending only, keyword "software")

{
  "sources": ["usaspending"],
  "keyword": "software",
  "max_results": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Example Input (All sources + custom URL)

{
  "sources": ["usaspending", "ted_europa", "uk_contracts"],
  "target_urls": ["https://procurement.agency.gov/tenders"],
  "max_results": 50
}

---

📤 Output Fields

Each tender is returned as a JSON object. Example output (USASpending contract):

[
  {
    "tender_id": "USA-019810652655",
    "title": "THE PRIMARY ACTIVITIES COVERED BY THIS EFFORT INCLUDE THE FINAL SERVICING MISSION...",
    "issuing_authority": "National Aeronautics and Space Administration",
    "recipient": "ASSOCIATION OF UNIVERSITIES FOR RESEARCH IN ASTRONOMY, INC.",
    "deadline": null,
    "posted_date": null,
    "value": "$2,511,478,245",
    "link": "https://www.usaspending.gov/award/GSFC0198106DNAS526555",
    "category": "IT / Technology",
    "naics_code": null,
    "location": "MD",
    "country": "United States",
    "description": "THE PRIMARY ACTIVITIES COVERED BY THIS EFFORT INCLUDE THE FINAL SERVICING MISSION...",
    "status": "Awarded",
    "source": "USASpending.gov",
    "processed_at": "2026-04-12T11:08:33.541764Z"
  }
]

Note: For TED Europa and UK Contracts Finder, the status is usually "Open" and recipient is omitted (not disclosed until award).

---

🛠 How to Use on Apify

1. Create a task with this actor.
2. Select sources – choose from USASpending, TED, UK, or add custom URLs.
3. Add filters – keyword, category, country, max results.
4. Configure proxy – optional (USASpending API rarely blocks; HTML scrapers may benefit from residential proxies).
5. Run – the actor fetches data from all selected sources.
6. Export – download results as JSON, CSV, or Excel.

Running via API

curl -X POST "https://api.apify.com/v2/acts/your-username~gov-tenders-scraper/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -d '{
    "sources": ["usaspending"],
    "keyword": "cloud",
    "max_results": 20
  }'

---

🎯 Use Cases

• Bid Tracking – Monitor new tenders from EU, UK, or US federal agencies.
• Competitor Intelligence – See which companies win large contracts (USASpending).
• Market Research – Analyse spending patterns by agency, category, or recipient.
• Business Development – Find active procurement opportunities in your industry.
• Compliance & Reporting – Gather data for contract audit or historical analysis.

---

❓ Frequently Asked Questions

Q1. Do I need an API key for USASpending.gov?

No. The USASpending API is completely open and free. No authentication required.

Q2. Why does SAM.gov not work?

SAM.gov is an Angular SPA that requires JavaScript rendering. This actor uses the underlying USASpending API, which provides the same data in pure JSON.

Q3. Can I scrape other government portals (e.g., Canada, Australia)?

Yes. Add the URL to target_urls. The actor will attempt to extract tenders using JSON‑LD, HTML cards, or link scanning. It works best with portals that use structured data.

Q4. What categories are available?

The actor detects 10 categories: IT / Technology, Construction, Energy, Healthcare, Consulting, Defence, Transport, Education, Environment, General Services. You can filter by any of these.

Q5. How fresh is the data?

• USASpending: Awarded contracts are usually updated within 30 days of signing. The API returns the most recent by value.
• TED & UK: Scrapes the live search results, so it is near real‑time for open tenders.

Q6. Do I need a proxy?

For USASpending: no, the API is very tolerant. For TED and UK Contracts Finder: not strictly required, but using residential proxies reduces the chance of being blocked during large runs.

Q7. Why are some fields null?

Not all sources provide every field (e.g., TED often lacks a precise deadline in the search result). The actor returns what is available.

Q8. How many results can I get per run?

The actor respects max_results and distributes it across sources. For USASpending, you can request up to 100 per call (API limit). For HTML scrapers, the limit is the number of visible results on the first page.

---

📝 Technical Notes

• Dependencies: curl_cffi (for TLS fingerprint spoofing) and BeautifulSoup4. Both are bundled.
• SSL: The actor disables SSL verification for government APIs (verify=False) due to certificate issues. This is safe for read‑only public data.
• Rate Limiting: The actor adds random delays (1–3 seconds) between source requests.
• Custom URL parsing: The actor first tries JSON‑LD, then HTML cards (using common class names), then falls back to link scanning.
• USASpending payload: Uses the /api/v2/search/spending_by_award/ endpoint with filters for contract award types (A,B,C,D).

---

🔍 SEO Keywords

government tenders scraper, USASpending API, TED Europa, UK Contracts Finder, procurement data extraction, federal contracts, bid opportunities, RFQ scraper, public sector tenders, Apify tender actor, contract award data, government procurement intelligence

---