CMS Open Payments Scraper - Pharma & Device Payments

Scrape federal Sunshine Act General, Research, and Physician Ownership payment disclosures from pharmaceutical, biotech, and medical device manufacturers to physicians and teaching hospitals via the official Centers for Medicare & Medicaid Services (CMS) Open Payments DKAN API.

Disclaimer: This Actor is an independent tool and is not affiliated with or endorsed by the Centers for Medicare & Medicaid Services (CMS) or the U.S. Department of Health and Human Services (HHS).

---

What it does

• Dynamic Dataset Resolution: Resolves the official CMS DKAN dataset UUID at runtime from https://openpaymentsdata.cms.gov/api/1/metastore/schemas/dataset/items by program year and payment type (General Payment Data, Research Payment Data, or Ownership Payment Data), never hardcoded.
• Manufacturer, State, Nature & Amount Filtering: Queries POST https://openpaymentsdata.cms.gov/api/1/datastore/query/<datasetId>/0 with server-side filters on recipient_state and the manufacturer name (and, for GENERAL with a single natureOfPayment term, a server-side LIKE on the nature column), plus client-side substring filtering on natureOfPayment and threshold filtering on minAmountUsd. Results are sorted newest first by the server (date_of_payment descending, or payment_publication_date for OWNERSHIP) before maxItems is applied. Within a single date, rows are ordered by amount descending; the date-level order is exact.
• OWNERSHIP field notes: CMS Ownership datasets do not publish a form of payment, a product, or a payment date, so formOfPayment, productCategory, and productName are null for OWNERSHIP rows, paymentDate carries the CMS publication date, dateBasis is "publicationDate" for OWNERSHIP rows (it is "paymentDate" for GENERAL and RESEARCH rows), amountUsd carries value_of_interest (falling back to total_amount_invested_usdollars), and manufacturer matching uses the submitting manufacturer name (the applicable manufacturer name is also populated on ownership records and matches the submitting name on sampled rows; the submitting name is the match key for consistency across payment types).
• Unified Schema Across Payment Types: Normalizes GENERAL, RESEARCH, and OWNERSHIP disclosures into a single 21-field output schema including npi, firstName, lastName, specialty, teachingHospitalName, manufacturerName, amountUsd, paymentDate, dateBasis, natureOfPayment, formOfPayment, productCategory, and productName.
• Newest-First Ordering: Deduplicates by recordId and sorts results by paymentDate descending (then amountUsd descending) before applying maxItems.

---

Who it's for

• Life Sciences Compliance & Legal Teams: Audit Sunshine Act disclosures, consulting fee distributions, and KOL (Key Opinion Leader) relationships across therapeutic areas.
• Medical Device & Biotech Commercial Analysts: Benchmark manufacturer spending, royalty/licensing arrangements, and clinical trial research payments by state and specialty.
• Healthcare Researchers & Journalists: Analyze financial relationships between drug/device manufacturers, teaching hospitals, and healthcare providers using primary CMS disclosures.

---

Input

All parameters have working defaults/prefills that return results in under 2 minutes.

Field	Type	Default / Prefill	Description
paymentType	string	"GENERAL"	GENERAL (consulting, royalties, food/beverage, travel, speaking), RESEARCH (clinical trial & research grants), or OWNERSHIP (physician ownership & investment interests).
programYear	integer	latest available	CMS Open Payments reporting program year (e.g. 2025, 2024, 2023). Leave blank or omit it to use the latest available year, resolved dynamically from the CMS metastore at runtime. An explicit year works exactly as before.
manufacturerNames	array<string>	["Stryker"]	Case-insensitive substring match against the paying manufacturer or GPO name (e.g. ["Stryker"], ["Medtronic", "Pfizer"]). Leave empty for all manufacturers.
natureOfPayment	array<string>	[]	Optional case-insensitive substring filters on natureOfPayment (e.g. ["Consulting", "Royalty"]). Leave empty for all payment natures. For RESEARCH rows the filter also matches the synthesized nature text, which includes the study name (for example "Research:
recipientStates	array<string>	["AZ"]	Two-letter US state or territory codes for the recipient (e.g. ["AZ"], ["AZ", "NV"]). Leave empty for all states.
minAmountUsd	number	0	Minimum payment amount (or ownership value of interest) in USD.
maxItems	integer	200	Maximum number of records to return.

Example input JSON

{
  "paymentType": "GENERAL",
  "programYear": 2025,
  "manufacturerNames": ["Stryker"],
  "natureOfPayment": [],
  "recipientStates": ["AZ"],
  "minAmountUsd": 0,
  "maxItems": 200
}

Example run options (with default $50 max charge protection)

When invoking via the Apify API or CLI, set a default maximum charge of $50 ("maxTotalChargeUsd": 50):

{
  "maxTotalChargeUsd": 50
}

The Actor checks calculate_max_event_charge_count_within_limit("apify-default-dataset-item") before every dataset push and stops cleanly when your spending limit is reached.

---

Output

Results are saved to the default dataset, deduplicated by recordId, and sorted by paymentDate descending.

Output fields

Field	Type	Description
recordId	string	CMS Open Payments unique record_id.
programYear	integer	Reporting program year (e.g. 2025).
paymentType	string	GENERAL, RESEARCH, or OWNERSHIP.
recipientType	string | null	Covered recipient classification (Covered Recipient Physician, Covered Recipient Teaching Hospital, etc.).
npi	string | null	10-digit National Provider Identifier (NPI) of the recipient or principal investigator.
firstName	string | null	Recipient provider first name.
lastName	string | null	Recipient provider last name.
specialty	string | null	Provider taxonomy / specialty string.
teachingHospitalName	string | null	Teaching hospital name when applicable.
city	string | null	Recipient practice city.
state	string | null	Recipient 2-letter state code.
zip	string | null	Recipient postal ZIP code.
manufacturerName	string | null	Applicable manufacturer or GPO making the payment.
amountUsd	number | null	Total payment amount (or ownership value of interest) in USD.
paymentDate	string | null	Date of payment in YYYY-MM-DD format. For OWNERSHIP rows this is the CMS publication date; see dateBasis.
dateBasis	string	What paymentDate represents: "publicationDate" for OWNERSHIP rows, "paymentDate" for GENERAL and RESEARCH rows.
natureOfPayment	string | null	Nature of payment or transfer of value (or study name / terms of interest).
formOfPayment	string | null	Form of payment or transfer of value (Cash or cash equivalent, In-kind items and services, etc.).
productCategory	string | null	Primary associated product category or therapeutic area.
productName	string | null	Primary associated drug, biological, device, or medical supply name.
sourceDatasetId	string	Resolved CMS DKAN dataset UUID.

Sample output row (synthetic example)

{
  "recordId": "9900112233",
  "programYear": 2025,
  "paymentType": "GENERAL",
  "recipientType": "Covered Recipient Physician",
  "npi": "1000000000",
  "firstName": "ALEX",
  "lastName": "EXAMPLE",
  "specialty": "Allopathic & Osteopathic Physicians|Orthopaedic Surgery",
  "teachingHospitalName": null,
  "city": "PHOENIX",
  "state": "AZ",
  "zip": "85004",
  "manufacturerName": "Example Medical Devices Corp.",
  "amountUsd": 1250.0,
  "paymentDate": "2025-11-14",
  "dateBasis": "paymentDate",
  "natureOfPayment": "Consulting Fee",
  "formOfPayment": "Cash or cash equivalent",
  "productCategory": "Orthopaedics",
  "productName": "Example Ortho System",
  "sourceDatasetId": "fb0b1734-1410-429d-92f6-3f4b35218e5e"
}

---

How pricing works

Pay per result: $3 per 1,000 rows returned ($0.003 per row). You only pay for rows returned.

---

Data source and terms

• Official Source: Centers for Medicare & Medicaid Services (CMS): Open Payments Data Portal
• Update Cadence: CMS Open Payments data is published annually (each new program year is released on or around June 30 and refreshed in January for dispute resolutions and corrections). It is an annual statutory disclosure dataset, not a real-time feed.
• Terms of Use: Published by CMS under the federal Physician Payments Sunshine Act (42 U.S.C. § 1320a-7h). See CMS Open Payments About the Data & Methodology.

---

Compliance

CMS Open Payments disclosures (including provider names, NPIs, specialties, and practice cities/states) are published by CMS under the Physician Payments Sunshine Act for public transparency, academic research, and regulatory compliance. CMS does not publish email addresses or telephone numbers in Open Payments, and this Actor does not perform third-party contact enrichment.

• CMS Data-Use Notes: Financial ties between healthcare providers and manufacturers do not by themselves indicate an improper relationship; consult the CMS Open Payments methodology when interpreting payment records.
• Outreach & Privacy Laws: Buyers are solely responsible for complying with the Telephone Consumer Protection Act (TCPA), the National Do-Not-Call (DNC) Registry, the CAN-SPAM Act, and all applicable state and international privacy laws before conducting any outreach.
• No Consumer / Individual Targeting: This tool is designed for healthcare compliance, transparency research, and B2B life sciences analysis, and must not be used for consumer targeting.

---

Canary input

A tiny self-test that returns rows in under 30 seconds: use the default input with "maxItems": 5. A healthy run ends with a status starting Complete: and 5 rows, newest first.

{
  "paymentType": "GENERAL",
  "programYear": 2025,
  "manufacturerNames": ["Stryker"],
  "natureOfPayment": [],
  "recipientStates": ["AZ"],
  "minAmountUsd": 0,
  "maxItems": 5
}

---

Troubleshooting

• Status starts with PARTIAL:: the run finished but something material was withheld or went wrong mid-scan (for example a CMS source error after some rows were collected, or invalid rows skipped). The status message states the reason and the row counts. Report repeated PARTIAL runs on the actor's Issues tab.
• Zero matches: a healthy run with no matching records ends Complete: 0 rows matched your filters, with the scanned count in the message. Widen manufacturerNames, add recipientStates, clear natureOfPayment, or lower minAmountUsd.
• Caps: reaching maxItems is reported in the Complete: message. Reaching your max charge limit ends the run as PARTIAL: with the reason stated; raise the limit to get more rows.
• Stale data: Open Payments is an annual dataset (new program year each June, refreshed in January). A run can never return payments newer than the latest published program year.

Found a problem or need a field? Open an issue on the Issues tab; we respond quickly.

---

Limits and FAQ

• What rate limit does the Actor use? The Actor throttles requests to under 3 requests/second (2.8 req/s) with 6 tries per request (5 automatic retries) and jittered exponential backoff from 2 to 32 seconds, capped at 120 seconds and honoring Retry-After, to respect CMS Open Payments infrastructure.
• How long do broad queries take? Leaving both manufacturerNames and recipientStates empty means all manufacturers and all states, as documented. That asks CMS to sort the full program-year table (about 15 million rows for GENERAL), so the first page alone can take well over a minute at the source. Expect a broad run to take several minutes (the run timeout is 900 seconds and the per-request read timeout is 120 seconds; broad queries are paged in small 25-row chunks because the full-table sort alone takes about two minutes at the source). Narrowing by manufacturer, state, or nature of payment is much faster: the default run typically finishes in seconds.
• Why does OWNERSHIP mode use publication date when date_of_payment is absent? CMS Ownership & Investment datasets report annual ownership stakes (value_of_interest and total_amount_invested_usdollars) along with payment_publication_date rather than a single transaction date; the Actor normalizes payment_publication_date into paymentDate.

---

Changelog

v0.1.3 (2026-11-01)

• Grok Red Team fix round. Medium: broad all-manufacturers plus all-states queries no longer die on the read timeout: the read timeout is 120 seconds, datastore pages after the first send "count": false, broad (unconditioned) queries page in 25-row chunks with the count off, a single GENERAL nature filter is also sent server-side, and the default run timeout is 900 seconds; the broad-query limitation and expected duration are documented in the FAQ. Medium: retry policy is now 6 tries with jittered backoff from 2 to 32 seconds, capped at 120 seconds, keeping Retry-After handling. Lows: an omitted programYear now defaults to the latest available year from the metastore instead of hardcoded 2025; source-caused hard-fail messages (the source API error and the 20 percent validation refusal) now carry the Issues CTA, while user-input errors and user-limit PARTIALs do not; OWNERSHIP rows carry dateBasis: "publicationDate" (other types carry "paymentDate"); the description and README intro start with "Scrape" and pricing states the per-1K figure; the final ordering is sorted client-side by the parsed payment date descending, so it no longer relies on the server text-column sort being single-year.

v0.1.2 (2026-10-31)

• Red Team fix round. High: paymentDate for GENERAL and RESEARCH no longer falls back to the publication date; a row with an absent or unparseable date_of_payment is now invalid and skipped (counted), never emitted with a publication date sorted as if it were a payment date. OWNERSHIP keeps the publication date by disclosure. Medium: the core-column sentinel now also covers the headline output columns (recipient type, names, specialty, city, zip, and, where the dataset carries them, form and product fields), so a rename there fails loudly instead of emitting silent nulls. Medium: the OWNERSHIP manufacturer note corrected (the applicable manufacturer name is populated and matches the submitting name on sampled rows). Lows: retry profile now honors Retry-After with jittered backoff; the within-date tiebreak (amount descending) and the RESEARCH study-name filter matching are documented; TEST_NOTES NPIs masked and its stale status quote bannered.

v0.1.1 (2026-10-31)

• Fixed newest-first ordering: sorting now happens server-side before paging, so the rows returned are the newest matches overall, not the newest of the first unordered page.
• Honest status contract: successful runs end Complete: (zero matches included), source problems mid-scan end PARTIAL: with the reason, and the Issues-tab call to action appears only on source-caused PARTIALs.
• Added a core-column sentinel and per-row validation before push; invalid rows are counted and disclosed.
• OWNERSHIP fixes: manufacturer matching uses the submitting manufacturer name, sorting uses the publication date, and formOfPayment is null (CMS publishes no form of payment for ownership records) instead of carrying the interest-holder classification.
• Store categories set to BUSINESS only; default run options now cap the max charge at $50.

v0.1 (2026-10-31)

• Initial release supporting CMS Open Payments GENERAL, RESEARCH, and OWNERSHIP datasets with dynamic DKAN metastore UUID resolution.