Senate Lobbying Disclosure (LDA) Scraper

Extract US Senate Lobbying Disclosure Act (LDA) filings with complete registrant, client, and financial data. This actor scrapes the official Senate LDA API to deliver structured lobbying intelligence for compliance, research, government affairs, and business development.

Features

• Comprehensive Lobbying Data: Extract registrant names, client information, income/expense data, issue codes, and filing dates
• Advanced Filtering: Filter by filing year, registrant name, client name, or specific lobbying issue codes
• Rate-Limited Access: Respects Senate API rate limits with automatic retry on throttling
• Structured Output: JSON records with 14 fields per filing, ready for analysis and database import
• High-Volume Capable: Fetch up to 2000+ filings per run with automatic pagination
• No Authentication Required: Scrapes public API data — no Senate account needed

Output Fields

Field	Type	Description
filingUuid	string	Unique identifier for the filing
filingType	string	Filing type (e.g., "Registration", "Amendment", "Termination")
filingYear	number	Year of the filing (e.g., 2025)
filingPeriod	string	Filing period (e.g., "1st Quarter", "2nd Quarter")
registrantName	string	Lobbying firm or individual registrant name
registrantState	string	Registrant state abbreviation (e.g., "NY", "CA")
clientName	string	Client company/organization name
clientState	string	Client state abbreviation
clientDescription	string	Client industry or business description
income	number	Total lobbying income reported (USD, null if not reported)
expenses	number	Total lobbying expenses reported (USD, null if not reported)
datePosted	string	Filing submission date (ISO 8601 format)
issueCodes	array	Array of lobbying issue codes (e.g., "BUD", "TAX", "ENV")
filingUrl	string	Direct link to the official Senate filing document

How to scrape US Senate LDA filings

Step 1: Access the Actor

Navigate to this actor in the Apify Store and click Run.

Step 2: Configure Your Search

In the Input tab, set your search parameters:

• Filing Year (required): Year of filings to extract. Default: 2025. Range: 2000–2100.
• Registrant Name Filter (optional): Partial match on lobbying firm name (e.g., "Lobbying Associates LLC").
• Client Name Filter (optional): Partial match on client name (e.g., "Microsoft" or "Apple").
• Issue Code Filter (optional): Specific lobbying issue code (e.g., "BUD" for Budget/Appropriations).
• Max Results (optional): Maximum filings to return. Default: 250 (safe for rate limits). Set to 0 for up to 2000 filings.

Step 3: Run the Actor

Click Run and monitor the execution:

• Actor logs show total filings available and pagination progress.
• Rate limiting is automatic — the actor waits if throttled (HTTP 429) and retries.
• Typical run time: 2–10 minutes depending on result count.

Step 4: Download Your Data

Once the run completes, download the results:

• JSON format for integration with databases, BI tools, or LLMs.
• CSV export for Excel or Google Sheets analysis.
• Results include all 14 fields per record.

Example Input

{
  "filingYear": 2025,
  "registrantFilter": "Jones Consulting",
  "clientFilter": "Microsoft",
  "issueCode": "TAX",
  "maxResults": 500
}

Example Output

{
  "filingUuid": "4f33e46d-4018-4899-8926-c03bb9977ae2",
  "filingType": "Registration",
  "filingYear": 2025,
  "filingPeriod": "1st Quarter (Jan 1 - Mar 31)",
  "registrantName": "COVENANT GOVERNMENT AFFAIRS, LLC",
  "registrantState": "FL",
  "clientName": "FUSE INTEGRATION, INC.",
  "clientState": "CA",
  "clientDescription": "Manufacturing communications systems",
  "income": null,
  "expenses": null,
  "datePosted": "2025-01-01T12:03:08-05:00",
  "issueCodes": ["BUD", "DFS"],
  "filingUrl": "https://lda.senate.gov/filings/public/filing/4f33e46d-4018-4899-8926-c03bb9977ae2/print/"
}

Pricing

$0.003 per record — Charged only for successfully extracted filings. A typical run of 250 filings costs ~$0.75 in actor compute units (plus Apify platform fees).

Cost estimation:

• 100 filings × $0.003 = $0.30
• 500 filings × $0.003 = $1.50
• 1000 filings × $0.003 = $3.00

Rate Limits & Performance

The Senate LDA API allows approximately 15 requests per minute for anonymous access. This actor:

• Automatically throttles requests to ~4.5 seconds per page
• Handles HTTP 429 (rate limit) responses by waiting 30 seconds and retrying
• Fetches 25 filings per request (page_size=25) for efficiency

Expected timing:

• 250 filings: ~2–3 minutes
• 500 filings: ~4–5 minutes
• 1000 filings: ~8–10 minutes

Data Freshness

Filings are typically posted to the Senate LDA website within 24 hours of submission. This actor queries the live API, so all data reflects the most recent filings available.

Other scrapers

Looking for more government and regulatory data?

• FARA Foreign Agents Scraper — Extract US Department of Justice FARA (Foreign Agents Registration Act) filings with agent and client data.
• FEC Candidates Finance Scraper — Scrape FEC candidate financial disclosures, contributions, and expenditure data.

FAQ

Q: Do I need a Senate account to use this? A: No. This actor queries the public Senate LDA API. No authentication is required.

Q: What is the data latency? A: Filings are typically posted within 24 hours. This actor uses the live API, so data is current.

Q: Can I filter by multiple criteria at once? A: Yes. Combine registrantFilter, clientFilter, and issueCode in the same run. Filters use partial matching (case-insensitive).

Q: What happens if I hit a rate limit? A: The actor automatically waits 30 seconds and retries. You should not encounter blocking errors under normal usage.

Q: Can I export to CSV or Excel? A: Yes. Use Apify's built-in export feature to download results as CSV, JSON, or Excel after the run completes.

Q: Why are some income/expense fields null? A: Initial registrations (RR filings) may not include financial data. Amendments (AM) and terminations (TR) may also be null if the registrant had zero reportable activity.

Q: How far back can I query? A: The API contains filings back to 2007 (earliest LDA disclosures). Set filingYear to any year from 2000–2100.

Legal Disclaimer

This actor scrapes data from the public US Senate Lobbying Disclosure Act (LDA) database at lda.senate.gov. All extracted data is publicly available government information subject to the public domain.

Use of this actor and its extracted data must comply with all applicable laws and regulations, including:

• The Lobbying Disclosure Act of 1995 (2 U.S.C. § 1601 et seq.)
• Terms of service of the Senate LDA website
• Applicable privacy and data protection laws

This actor is provided for informational and research purposes only. The publisher assumes no liability for misuse or unlawful application of extracted data. Users are solely responsible for compliance with all legal requirements.