NY Lobbying Search Scraper

Scrape New York State's official Lobbying Search application (reports.ethics.ny.gov), run by the NYS Commission on Ethics and Lobbying in Government. Search lobbyist/client relationships, subjects and parties lobbied, and every disclosure filing type — source of funding, disbursement of public monies, reportable business relationships, and coalition contributions. No login, no API key, no proxy required.

What this actor does

• 7 search modes, matching every tab of the official public search tool:
  • globalSearch — lobbyist / client / public corporation / coalition relationships
  • subjectsLobbied — filings by subject-matter category (health, tax, energy, ...)
  • partiesLobbied — filings by government body lobbied (agencies, legislature, municipalities, ...)
  • sourceOfFunding — source-of-funding disclosure statements
  • disbursementOfPublicMonies — disbursement-of-public-monies disclosures
  • reportableBusinessRelationship — reportable business relationship disclosures
  • coalitionContribution — coalition contribution disclosures
• Rich nested detail — principal lobbyists, contractual/beneficial clients, and individual lobbyists with profile links, addresses, and entity types
• Dollar amounts parsed to numbers — compensation, expenses, and contribution fields are emitted as plain numeric USD values, not currency strings
• Direct filing links — every record includes a link to its official filing detail page on the Commission's site
• Empty fields are always omitted — every record only contains data that was actually reported

Output per record

Every record shares this core (present regardless of mode):

• formSubmissionId, filingYear, filingType, filingPeriod, filingConfirmationNumber, filingStatus
• isPublicCorporation
• principalLobbyists[], contractualClients[], beneficialClients[] — each with id, name, filedAsName (if amended), entityType, address
• individualLobbyists[] — firstName/lastName or organizationName, role, profileUrl
• subjectsLobbied[] — subject-matter categories reported on the filing
• associatedFilingUrl — link to the official filing detail page
• sourceUrl — canonical record URL (same as associatedFilingUrl when available)
• recordType — the search mode that produced the record
• scrapedAt — UTC timestamp

Mode-specific fields layer on top:

Mode	Extra fields
subjectsLobbied	totalCompensationExpensesUsd, totalExpensesUsd
partiesLobbied	activityType, communicationType, governmentBody, partyLobbiedName, lobbyingFocus, lobbyingFocusNumber
sourceOfFunding	sourceType, sourceName, dateOfContribution, contributionAmountUsd, contractualClientName, contractualClientAddress, sofReportingRequired, sofReasonNotRequired, sourceOfFundingClients[]
disbursementOfPublicMonies	descriptionsOfActivities[], compensationUsd, reimbursedExpensesUsd, noExpensesReported
reportableBusinessRelationship	filerName, filerId, relationshipType, relationshipWith, statePersonInvolved, employedBy, paymentMade, servicesProvided, compensationAmountUsd, reimbursedExpenseAmountUsd
coalitionContribution	coalitionName, coalitionMember, contributionAmountUsd, expenseIncurredOnBehalf, totalExpenseAmountUsd, pooledFundsUsed, pooledFundsMemberPortionUsd, totalContributionAmountUsd, expensePurpose

Input

Field	Type	Applies to	Description
mode	select	all	Search mode (see above)
searchQuery	string	all	Individual/organization name or address to search for
year	select	all	Filing year, or ALL
filerType	select	globalSearch	Lobbyist / Client / Public Corporation / Coalition / ALL
levelOfGovernment	select	globalSearch, disbursementOfPublicMonies	State / Municipal / Both / ALL
filerCategory	select	subjectsLobbied, partiesLobbied	Lobbyist / Client / Public Corporation / Coalition / ALL
subject	select	subjectsLobbied	Fine-grained subject-matter category (93 categories sourced live from the site's own taxonomy — e.g. Cannabis/Marijuana, Cryptocurrency, Health – Pharmaceuticals, or ALL)
period	select	subjectsLobbied, disbursementOfPublicMonies	Bi-monthly filing period, or ALL
filingTypeFilter	select	subjectsLobbied, partiesLobbied	Registration / Bi-Monthlies / Client Semi-Annuals / ALL
partyType	select	partiesLobbied	Government body category (18 options, or ALL)
specificAgency	select	partiesLobbied, partyType=AGCY	Narrow to one specific state agency/authority (110 options, or ALL) — only applied when partyType is AGCY
communicationType	select	partiesLobbied	Direct / Grassroots / Direct+Grassroots / Monitoring Only / ALL
sourceType	select	sourceOfFunding, coalitionContribution	Single / Multiple / Not Required to Report / ALL
relationshipType	select	reportableBusinessRelationship	Entity / State Person / ALL
employedBy	select	reportableBusinessRelationship	Assembly / Executive Chamber / Senate / State Agency / ALL
maxItems	integer	all	Hard cap on emitted records (1–2000)

Example: search Global Search for a company name

{
  "mode": "globalSearch",
  "searchQuery": "Google",
  "year": "ALL",
  "maxItems": 50
}

Example: browse Health-subject lobbying filings

{
  "mode": "subjectsLobbied",
  "subject": "37",
  "year": "2024",
  "maxItems": 100
}

Example: filings lobbying a state agency

{
  "mode": "partiesLobbied",
  "partyType": "AGCY",
  "year": "2024",
  "maxItems": 100
}

Example: coalition contribution disclosures for a year

{
  "mode": "coalitionContribution",
  "year": "2024",
  "maxItems": 100
}

Use cases

• Investigative journalism — trace which lobbyists represent which clients, and which state agencies or legislative bodies they target
• Government transparency / policy research — analyze lobbying spend by subject matter (health, energy, gaming, ...) over time
• Compliance & due diligence — check a company's or law firm's registered lobbying relationships and disclosures
• Academic research — bulk-export New York's lobbying disclosure corpus for quantitative analysis
• Competitive intelligence — monitor which organizations are actively lobbying on issues relevant to your industry

FAQ

Do I need a login or API key? No. This scrapes the public, no-auth search endpoints that power reports.ethics.ny.gov/publicquery.

Why do some records for the same filing repeat? A single filing can report multiple distinct facts — e.g. lobbying two different state agencies, or making two separate coalition contributions in the same period. Each is emitted as its own record; formSubmissionId links records that share a filing.

What does filingYear like 2025 - 2026 mean? Lobbyist/client registrations run on a two-year legislative session; the value reflects the session the registration covers. Use the year input filter to match either year in the range.

Are dollar amounts accurate? They're parsed directly from the Commission's own reported figures — the actor doesn't recompute or estimate.

How far back does the data go? The public search covers filings from 2019 onward.

What if my filter combination returns 0 results? The actor reports a status message and exits cleanly — try widening year to ALL or removing other filters.

Can I filter Parties Lobbied by State vs. Municipal activity level? No — the Commission's own public search tool exposes a "State/Municipal activity" dropdown, but it's broken on their end: selecting anything other than the default returns zero results, confirmed directly against their live search API independent of this actor. Rather than expose a filter that can never return data, the actor always queries unfiltered on this axis (equivalent to the "ALL" state).

Can I narrow Parties Lobbied to a specific senator, assembly member, or municipality (county/city/town/village)? Not currently. The official site's own drill-down selectors for a specific senator/assembly-member/executive-office and for municipal entities (counties, cities, towns, villages, school districts, etc.) either don't reliably filter server-side when queried directly (confirmed by testing — the "specific senator" filter silently returns the unfiltered result set) or cover thousands of unstable entity names (e.g. 900+ towns, 7,000+ special taxing districts) that can't be maintained as a reliable dropdown. The one drill-down that does filter reliably — narrowing to a specific state agency — is exposed via specificAgency (only applies when partyType is AGCY).

Why does coalitionContribution not have an associatedFilingUrl per record? The upstream Coalition Contribution endpoint doesn't expose a per-row filing-detail link — sourceUrl falls back to the main search application URL for this mode only. Every other mode links directly to the filing.

A very broad partiesLobbied query with partyType: ALL combined with another filter (like communicationType) is slow or returns nothing — why? Some very broad filter combinations trigger a genuinely slow server-side query on the state's own site (independently confirmed to take 90s+ regardless of how few rows are requested). The actor fails soft with a status message rather than hanging — narrow the query by picking a specific partyType (or subject, for subjectsLobbied) instead of ALL.