Government Budget & Spending Notices Scraper

💰 Government Budget & Spending Notices Scraper is a powerful Apify Actor designed to discover, track, and collect federal award data directly from USASpending.gov's own public API. This tool provides comprehensive Government Budget & Spending intelligence including award ID, recipient/title, award amount, start date, and a direct link to the official award record for any keyword. Whether you're building a public spending dashboard, tracking federal award trends, or conducting government transparency research, the Government Budget & Spending Notices Scraper delivers actionable Government Budget & Spending insights efficiently.

With grouped award-type querying, direct JSON API access, resilient error handling, and reliable Apify Dataset delivery, the Government Budget & Spending Notices Scraper ensures comprehensive Government Budget & Spending coverage across both contracts and grants/assistance award types. It focuses on key Government Budget & Spending signals including award amount, recipient, and award date, making it an essential tool for public spending research and transparency intelligence.

---

📋 Table of Contents

• Features
• Data Source
• How It Works
• Input
• Output
• Technical Stack
• Data Fields
• Use Cases
• Quick Start
• Configuration
• Performance
• Important Notes
• License & Legal

---

🔥 Features

• Direct API Access – Queries USASpending.gov's official spending_by_award API directly for fast, structured Government Budget & Spending data.
• Dual Award-Type Coverage – Searches both contracts (types A–D) and grants/assistance (types 02–05) in a single run.
• Keyword-Based Search – Scans Government Budget & Spending records for any keyword across a configurable multi-year time period.
• Amount-Sorted Results – Returns results sorted by award amount, descending, so the largest awards surface first.
• Direct Award Links – Builds a clickable link to the official USASpending.gov award record for every result.
• Residential Proxy Support – Optional Apify proxy for reliable Government Budget & Spending API access.
• Configurable Result Cap – Control how many records are pulled per award-type group.
• Real-Time Dataset Push – Pushes all collected Government Budget & Spending records to Apify Dataset in a single batch.
• Graceful Error Handling – Logs HTTP, validation, and JSON parsing errors per group without stopping the whole run.

---

📊 Data Source

USASpending.gov Public API

• Authority: The official U.S. government spending transparency API (api.usaspending.gov)
• Access Method: Direct async HTTP POST requests via httpx
• Coverage: Any publicly listed Government Budget & Spending award — contracts and grants/assistance — matching the given keyword
• Data: Structured JSON award records covering fiscal years 2020–2026 by default
• Access: Public API, no API key required
• Update Frequency: Reflects USASpending.gov's current published award data at time of run

---

⚙️ How It Works

The Government Budget & Spending Notices Scraper accepts a search keyword and queries USASpending.gov's spending_by_award API twice — once for contract award types and once for grant/assistance award types — so both major categories of federal spending are covered. Each query requests the top results sorted by award amount in descending order, within a fixed 2020–2026 time window. The Actor parses each result into a clean record containing the award ID, a title (description or recipient name), amount, start date, and a direct link to the award's USASpending.gov page. Every successfully parsed Government Budget & Spending record is collected and pushed to the Apify Dataset in a single batch at the end of the run.

Key Processing Steps:

1. Input Parsing – Accept a search keyword, max results per group, and proxy setting
2. Proxy Setup – Configure Apify proxy for the HTTP client, if enabled
3. Award-Type Group Loop – Iterate through contracts and grants/assistance type groups
4. API Request – POST a search query to USASpending.gov's spending_by_award endpoint
5. Error Handling – Catch and log HTTP, validation, and JSON parsing errors per group
6. Result Parsing – Extract award ID, title, amount, date, and build the award link
7. Group Aggregation – Combine results from both award-type groups into one list
8. Dataset Push – Push all collected records from the run in a single batch

Key Benefits:

• Pull Government Budget & Spending award data for any keyword without manual searching
• Cover both contracts and grants/assistance in a single run
• Surface the largest federal awards first with amount-based sorting
• Feed public spending dashboards, journalism research, or transparency tools
• Automate recurring Government Budget & Spending checks as new awards are published

---

📥 Input

The Actor accepts the following input parameters:

Field	Type	Default	Description
keyword	string	(required)	Search term to scan for matching Government Budget & Spending awards.
max_results	integer	50	Maximum number of records to request per award-type group.
useApifyProxy	boolean	true	Enable Apify proxy for the API requests.

Example Input:

{
  "keyword": "solar energy",
  "max_results": 50,
  "useApifyProxy": true
}

Search a Single Keyword:

{
  "keyword": "rural broadband"
}

Pull a Larger Batch:

{
  "keyword": "disaster relief",
  "max_results": 100
}

---

📤 Output

The Actor pushes Government Budget & Spending records with the following structure:

Field	Type	Description
doc_id	string	The official Award ID from USASpending.gov
title	string	Award description, or recipient name if no description is available
amount	string/number	Total award amount as reported by USASpending.gov
link	string	Direct URL to the award's official USASpending.gov page
date	string	Award start date

Example Government Budget & Spending Record:

{
  "doc_id": "SPE30020D1234",
  "title": "Rural Broadband Infrastructure Expansion Grant",
  "amount": "18500000",
  "link": "https://www.usaspending.gov/award/CONT_AWD_1234567_9700_...",
  "date": "2026-03-12"
}

---

🧰 Technical Stack

• HTTP Client: httpx (async) for fast, direct requests to USASpending.gov's API
• Concurrency: asyncio for non-blocking, multi-group querying
• Data Format: JSON request/response handling for the spending_by_award endpoint
• Proxy: Apify Proxy (optional), wired into the httpx.AsyncClient
• Logging: Apify Actor logging system
• Platform: Apify Actor serverless environment

---

📊 Data Fields Explained

Award Identity

• doc_id: The official Award ID assigned by USASpending.gov
• title: The award's description, falling back to the recipient name when no description exists

Financial Details

• amount: The total dollar value of the Government Budget & Spending award
• date: The award's start date

Access

• link: Direct link to view the full award record on the official USASpending.gov website

---

🎯 Use Cases

• Public Spending Research – Track how federal money is being awarded for a given keyword or industry
• Journalism & Transparency – Source primary award data for government spending stories
• Competitor Tracking – Monitor which organizations are winning Government Budget & Spending awards in a sector
• Nonprofit & Grant Research – Identify recent grant activity related to a specific cause or region
• Academic Research – Study federal spending patterns at scale
• Market Research – Gauge government investment trends across industries and time periods

---

🚀 Quick Start

1. Prepare Input

Go to Apify Console and enter:

{
  "keyword": "solar energy"
}

2. Run the Actor

Click Start. The Actor will:

• Query USASpending.gov for matching contract awards
• Query USASpending.gov for matching grant/assistance awards
• Parse and combine results from both groups
• Push all results to the Dataset

3. Monitor Progress

Console shows:

[USASpending] Searching 'solar energy' in contracts...
 -> 42 record(s) from contracts.
[USASpending] Searching 'solar energy' in grants_and_assistance...
 -> 18 record(s) from grants_and_assistance.
Done! Successfully saved 60 record(s) to Apify Dataset.

4. View & Download Results

• Results Tab: All Government Budget & Spending records
• Export: JSON, CSV, Excel, or HTML
• Filter: By amount or award date
• API Access: Available via the Apify API

---

⚙️ Configuration

Single keyword search:

{
  "keyword": "rural broadband"
}

Larger batch pull:

{
  "keyword": "disaster relief",
  "max_results": 100
}

Run without proxy:

{
  "keyword": "solar energy",
  "useApifyProxy": false
}

---

📈 Performance

Processing Speed

• Two API requests per run — one for contracts, one for grants/assistance
• No browser rendering required, so requests complete quickly
• Response time depends primarily on USASpending.gov's own API latency

Resource Usage

• Memory: Low, since only lightweight JSON requests are made
• Network: Two POST requests per run, regardless of keyword result volume
• Proxy: One shared Apify proxy connection across both requests, if enabled

---

⚠️ Important Notes

Legal & Compliance

• Fair Use: Uses USASpending.gov's official public API as intended
• Public Data Only: Retrieves only publicly published Government Budget & Spending award data
• Legal: Not legal advice — consult qualified professionals before using this data for compliance-sensitive decisions

Data Quality

• Freshness: Reflects USASpending.gov's currently published data at time of run
• Completeness: Fixed to a 2020–2026 time window by default; results depend on what has been reported to USASpending.gov
• Accuracy: Sourced directly from the official U.S. government spending transparency API
• Verification: Cross-check high-stakes figures against the live USASpending.gov website

Best Practices

• Use specific keywords to keep result sets relevant and manageable
• Increase max_results if you need deeper coverage per award-type group
• Re-run periodically to catch newly published awards
• Expect occasional 422 validation errors to be logged and skipped rather than stopping the run
• Combine with agency- or NAICS-specific research tools for a fuller Government Budget & Spending picture

---

📄 License & Legal

Terms of Use:

• Use for legitimate research, journalism, and transparency purposes
• Respect USASpending.gov's Terms of Use
• Use Government Budget & Spending data responsibly and in compliance with applicable laws

Disclaimer: Government Budget & Spending Notices Scraper is provided as-is for research and analytics purposes. Users are responsible for compliance with USASpending.gov's terms and all applicable laws. This is not legal advice.

---

⚖️ Government Budget & Spending Excellence

This Actor is optimized for Government Budget & Spending research with:

• ✅ Direct, official API access — no scraping fragility
• ✅ Dual coverage of contracts and grants/assistance
• ✅ Amount-sorted results for fast prioritization
• ✅ Resilient per-group error handling
• ✅ Real-time Dataset push
• ✅ Production-ready code