# Senate Lobbying Disclosure Scraper (`automation-lab/senate-lobbying-disclosure-scraper`) Actor Scrape LDA.gov Senate lobbying disclosures: registrants, clients, lobbyists, issues, amounts, filing dates, and official document links. - **URL**: https://apify.com/automation-lab/senate-lobbying-disclosure-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Senate Lobbying Disclosure Scraper Scrape public U.S. Senate Lobbying Disclosure Act (LDA) filings from LDA.gov. Use this actor to collect lobbying registrations, quarterly activity reports, registrants, clients, lobbyists, issue areas, filing dates, reported amounts, and official filing document links. ### What does Senate Lobbying Disclosure Scraper do? Senate Lobbying Disclosure Scraper extracts structured filing records from the official public LDA.gov filings API. It is built for repeat monitoring and research workflows where you need clean rows instead of clicking through the public search form. The actor returns one dataset item per filing. Each item includes the filing UUID, report type, report period, registrant, client, lobbyists, issue codes, issue descriptions, posted date, reported income or expenses, and source links. ### Who is it for? - ๐Ÿงพ Compliance teams tracking lobbying disclosures for regulated industries. - ๐Ÿ›๏ธ Public-affairs firms monitoring clients, registrants, lobbyists, and issue areas. - ๐Ÿ“ฐ Journalists researching lobbying activity by company, industry, or policy topic. - ๐Ÿ“Š B2B analysts enriching market-intelligence datasets with lobbying spend and policy interests. - ๐Ÿง‘โ€โš–๏ธ Advocacy and legal researchers reviewing official public filings. ### Why use this actor? The LDA.gov website is public, but manual searches are slow when you need many filings. This scraper automates the official search workflow and outputs JSON, CSV, Excel, or API-ready data through Apify datasets. It also keeps source URLs in each row so analysts can audit findings back to the official record. ### Data fields | Field | Description | | --- | --- | | `filingUuid` | Official LDA filing UUID | | `filingTypeDisplay` | Human-readable filing type | | `filingYear` | Filing/report year | | `filingPeriodDisplay` | Human-readable report period | | `registrantName` | Lobbying registrant | | `clientName` | Lobbying client | | `amountReported` | Income or expenses reported on the filing | | `postedDate` | Date/time posted on LDA.gov | | `issueCodes` | LDA issue area codes | | `issueNames` | Issue area names | | `issueDescriptions` | Detailed lobbying issue text | | `lobbyistNames` | Lobbyists listed in activities | | `governmentEntities` | Government entities listed in activities | | `sourceUrl` | Public filing page URL | | `documentUrl` | Official print/document URL | | `apiUrl` | Official LDA API URL | | `scrapedAt` | Scrape timestamp | ### How much does it cost to scrape Senate lobbying disclosures? This actor uses pay-per-event pricing, so costs scale with the number of filing records you save. - **Run start:** $0.005 once per run. - **Per filing item:** tiered by Apify plan. The BRONZE reference price is **$0.000031907 per filing**; FREE is **$0.000036693**, SILVER is **$0.000024888**, GOLD is **$0.000019144**, PLATINUM is **$0.000012763**, and DIAMOND is **$0.00001**. - **Small research run:** 100 filings on BRONZE costs about **$0.0082** ($0.005 start + 100 ร— $0.000031907). - **Monitoring run:** 1,000 filings on BRONZE costs about **$0.0369** ($0.005 start + 1,000 ร— $0.000031907). - **Free-plan estimate:** 100 filings costs about **$0.0087** ($0.005 start + 100 ร— $0.000036693). The pricing was set from realistic cloud-run cost evidence, with the 101-item launch validation run keeping an estimated net margin above 88% while staying far below comparable Senate/LDA scraper PPE benchmarks. ### Input options You can filter filings by registrant name, client name, lobbyist name, filing type, filing year, filing period, issue codes, and posted date range. Use `maxItems` to cap the number of filing records saved. Use `includePdfLinks` to include official LDA print/document URLs. Use `includeIssueDetails` to include detailed policy issue text. ### Example input ```json { "registrantName": "Google", "filingTypes": ["Q1", "Q2", "Q3", "Q4"], "filingYear": 2024, "maxItems": 25, "includePdfLinks": true, "includeIssueDetails": true } ```` ### Example output ```json { "filingUuid": "d715cb88-4878-4ee3-a1dc-b9aed7136aed", "filingTypeDisplay": "1st Quarter - Report", "filingYear": 2024, "registrantName": "GOOGLE CLIENT SERVICES LLC", "clientName": "GOOGLE CLIENT SERVICES LLC", "amountReported": 3080000, "issueCodes": ["ADV", "CPT", "DEF"], "sourceUrl": "https://lda.senate.gov/filings/public/filing/d715cb88-4878-4ee3-a1dc-b9aed7136aed/" } ``` ### How to run 1. Open the actor on Apify. 2. Enter a registrant, client, lobbyist, or filing filter. 3. Set `maxItems` for the number of filings you need. 4. Start the run. 5. Export the dataset as JSON, CSV, Excel, XML, RSS, or HTML. ### Tips for best results - Start with a registrant or client name for focused monitoring. - Add a filing year to keep runs fast and targeted. - Use filing types `Q1`, `Q2`, `Q3`, and `Q4` for quarterly activity reports. - Use `RR` and `RA` for registrations and registration amendments. - Keep `includeIssueDetails` enabled when doing policy-topic analysis. ### Filing type examples - `RR` โ€” Registration - `RA` โ€” Registration amendment - `Q1` โ€” 1st Quarter report - `Q2` โ€” 2nd Quarter report - `Q3` โ€” 3rd Quarter report - `Q4` โ€” 4th Quarter report - `YY` โ€” Year-end report - `MM` โ€” Mid-year report ### Issue code examples - `TEC` โ€” Telecommunications - `TAX` โ€” Taxation/Internal Revenue Code - `HCR` โ€” Health Issues - `DEF` โ€” Defense - `TRD` โ€” Trade - `SCI` โ€” Science/Technology - `CPT` โ€” Copyright/Patent/Trademark ### Integrations Send new lobbying records to Google Sheets for analyst review. Load filings into a warehouse for compliance dashboards. Trigger Slack alerts when a watched company files a new quarterly activity report. Join LDA records with CRM account lists for account intelligence. Feed official source links into a document review workflow. ### API usage with Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/senate-lobbying-disclosure-scraper').call({ registrantName: 'Google', filingYear: 2024, maxItems: 25 }); console.log(run.defaultDatasetId); ``` ### API usage with Python ```python from apify_client import ApifyClient client = ApifyClient('YOUR_APIFY_TOKEN') run = client.actor('automation-lab/senate-lobbying-disclosure-scraper').call(run_input={ 'registrantName': 'Google', 'filingYear': 2024, 'maxItems': 25, }) print(run['defaultDatasetId']) ``` ### API usage with cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab~senate-lobbying-disclosure-scraper/runs?token=$APIFY_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"registrantName":"Google","filingYear":2024,"maxItems":25}' ``` ### MCP for Claude Code and Desktop Use the Apify MCP server with this actor enabled: ```text https://mcp.apify.com/?tools=automation-lab/senate-lobbying-disclosure-scraper ``` Add it in Claude Code: ```bash claude mcp add apify-lobbying "https://mcp.apify.com/?tools=automation-lab/senate-lobbying-disclosure-scraper" ``` Desktop JSON configuration example: ```json { "mcpServers": { "apify-lobbying": { "url": "https://mcp.apify.com/?tools=automation-lab/senate-lobbying-disclosure-scraper" } } } ``` Example prompts showing MCP usage: - "Use the Senate Lobbying Disclosure Scraper to scrape 2024 quarterly lobbying filings for Google and summarize the top issue areas." - "Use the Apify MCP tool to find filings for a client named Microsoft and return document links." - "Run the lobbying filing scraper through MCP to monitor telecom issue code TEC filings for 2024 and export rows to a table." ### Legality ### Legal and ethical use This actor collects public records from the official U.S. lobbying disclosure system. You are responsible for using the data in accordance with applicable laws, platform terms, and your organization's compliance rules. Always keep source URLs when publishing or citing findings. ### FAQ ### Troubleshooting #### Why did I get fewer filings than expected? Check your filters. Filing type, filing period, issue codes, and date filters can narrow results quickly. #### Why is `amountReported` empty? Some registration or no-activity filing types may not report income or expenses. #### Why are document links not present? Set `includePdfLinks` to `true` to include official filing document URLs. ### Related scrapers - https://apify.com/automation-lab/doj-fara-filings-scraper - https://apify.com/automation-lab/sec-form-adv-filing-monitor - https://apify.com/automation-lab/house-financial-disclosures-scraper - https://apify.com/automation-lab/govtrack-congress-scraper ### Changelog Initial version extracts public LDA filing records from the official LDA.gov API. ### Support Open an Apify issue if a run fails or a field changes on LDA.gov. Include your run ID and input JSON so the issue can be reproduced. ### Notes on data freshness LDA.gov is the authoritative source for these public filings. The actor returns data available from the API at run time. Schedule recurring runs to monitor new filings over time. ### Operational workflow For recurring compliance monitoring, create a saved task with a company or issue filter. Run it daily, weekly, or after known filing deadlines. Use the Apify dataset API to pull only the latest run output. ### Export formats Apify datasets support JSON, JSONL, CSV, Excel, XML, RSS, and HTML exports. CSV and Excel are useful for analyst review. JSON and JSONL are useful for automation and warehouse pipelines. ### Source transparency Every record includes `sourceUrl` and `apiUrl`. When `includePdfLinks` is enabled, every available record also includes `documentUrl`. These links make audit and verification straightforward. ### Performance The actor uses the official API directly and does not need a browser. Most focused searches finish quickly. Large broad searches are paginated and should use `maxItems` to control volume. # Actor input Schema ## `registrantName` (type: `string`): Lobbying registrant name to search, for example Google, Amazon, AARP, or a Senate registrant ID. ## `clientName` (type: `string`): Optional lobbying client name to search. ## `lobbyistName` (type: `string`): Optional lobbyist name to search. ## `filingTypes` (type: `array`): Optional LDA filing type codes. Common values: RR registration, RA registration amendment, Q1-Q4 quarterly reports, YY year-end report. ## `filingYear` (type: `integer`): Optional filing year filter. ## `filingPeriod` (type: `string`): Optional filing period filter used by the LDA API. ## `postedFrom` (type: `string`): Optional client-side lower bound for posted date, in YYYY-MM-DD format. ## `postedTo` (type: `string`): Optional client-side upper bound for posted date, in YYYY-MM-DD format. ## `issueCodes` (type: `array`): Optional LDA issue codes such as TEC, TAX, HCR, DEF, TRD, or SCI. ## `includePdfLinks` (type: `boolean`): Include LDA filing print/document URLs in the output. ## `includeIssueDetails` (type: `boolean`): Include detailed lobbying issue descriptions. Turn off for smaller output rows. ## `maxItems` (type: `integer`): Maximum number of filing records to save. ## Actor input object example ```json { "registrantName": "Google", "filingTypes": [ "Q1", "Q2", "Q3", "Q4" ], "filingYear": 2024, "filingPeriod": "", "includePdfLinks": true, "includeIssueDetails": true, "maxItems": 20 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "registrantName": "Google", "filingTypes": [ "Q1", "Q2", "Q3", "Q4" ], "filingYear": 2024, "includePdfLinks": true, "includeIssueDetails": true, "maxItems": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/senate-lobbying-disclosure-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "registrantName": "Google", "filingTypes": [ "Q1", "Q2", "Q3", "Q4", ], "filingYear": 2024, "includePdfLinks": True, "includeIssueDetails": True, "maxItems": 20, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/senate-lobbying-disclosure-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "registrantName": "Google", "filingTypes": [ "Q1", "Q2", "Q3", "Q4" ], "filingYear": 2024, "includePdfLinks": true, "includeIssueDetails": true, "maxItems": 20 }' | apify call automation-lab/senate-lobbying-disclosure-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/senate-lobbying-disclosure-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Senate Lobbying Disclosure Scraper", "description": "Scrape LDA.gov Senate lobbying disclosures: registrants, clients, lobbyists, issues, amounts, filing dates, and official document links.", "version": "0.1", "x-build-id": "95c243af2fYKsN6fS" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~senate-lobbying-disclosure-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-senate-lobbying-disclosure-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~senate-lobbying-disclosure-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-senate-lobbying-disclosure-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~senate-lobbying-disclosure-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-senate-lobbying-disclosure-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "registrantName": { "title": "Registrant name or Senate ID", "type": "string", "description": "Lobbying registrant name to search, for example Google, Amazon, AARP, or a Senate registrant ID." }, "clientName": { "title": "Client name", "type": "string", "description": "Optional lobbying client name to search." }, "lobbyistName": { "title": "Lobbyist name", "type": "string", "description": "Optional lobbyist name to search." }, "filingTypes": { "title": "Filing types", "type": "array", "description": "Optional LDA filing type codes. Common values: RR registration, RA registration amendment, Q1-Q4 quarterly reports, YY year-end report.", "items": { "type": "string" } }, "filingYear": { "title": "Filing year", "minimum": 1999, "maximum": 2030, "type": "integer", "description": "Optional filing year filter." }, "filingPeriod": { "title": "Filing period", "enum": [ "", "first_quarter", "second_quarter", "third_quarter", "fourth_quarter", "mid_year", "year_end" ], "type": "string", "description": "Optional filing period filter used by the LDA API.", "default": "" }, "postedFrom": { "title": "Posted from", "type": "string", "description": "Optional client-side lower bound for posted date, in YYYY-MM-DD format." }, "postedTo": { "title": "Posted to", "type": "string", "description": "Optional client-side upper bound for posted date, in YYYY-MM-DD format." }, "issueCodes": { "title": "Issue area codes", "type": "array", "description": "Optional LDA issue codes such as TEC, TAX, HCR, DEF, TRD, or SCI.", "items": { "type": "string" } }, "includePdfLinks": { "title": "Include filing document links", "type": "boolean", "description": "Include LDA filing print/document URLs in the output.", "default": true }, "includeIssueDetails": { "title": "Include issue descriptions", "type": "boolean", "description": "Include detailed lobbying issue descriptions. Turn off for smaller output rows.", "default": true }, "maxItems": { "title": "Maximum filings", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of filing records to save.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```