🏛️ Socrata Open Data Scraper: CDC, NYC & Gov Data (No Key)

Overview

Socrata Open Data Scraper searches and pulls data from any Socrata-powered open-data portal: data.cdc.gov, datahub.hhs.gov, data.cms.gov, data.ny.gov, data.cityofnewyork.us, data.texas.gov, plus hundreds more US federal, state, and city portals. Two modes in one actor: discover datasets across every portal by keyword, then pull the rows you want with SoQL filters. Clean structured JSON, no API key, no anti-bot friction.

Socrata powers a huge share of US government open data and exposes two excellent public APIs: a cross-portal Discovery API and the per-dataset SODA query API with SoQL filtering. This actor wraps both, so you go from "I need X data" to clean rows in a single run.

Reliability posture: a record is only charged after it has actually been pushed to the dataset. Blocked requests, empty searches, and failed pulls are never charged.

✅ No API key | ✅ Hundreds of portals | ✅ Full SoQL filtering | ✅ First 10 records free | ✅ MCP-ready for AI agents

Features

Discovery mode: search all Socrata portals by keyword and get back matching datasets with domain, ID, name, and description. Data mode: point at a domain plus dataset ID and pull rows with full SoQL support ($where, $select, $order). Hundreds of portals covered: federal (CDC, HHS, CMS), state (NY, Texas, Washington), and city (NYC, LA, Seattle, Chicago). Optional Socrata app token for higher rate limits on very large pulls; keyless by default. Fully open (Socrata's own APIs), so there is no anti-bot, no cookie handling, and no login.

How it works

In discovery mode, leave domain and datasetId blank and pass a searchQuery. The actor calls Socrata's Discovery API (api.us.socrata.com/api/catalog/v1) with your keyword and returns matching datasets from every portal.

In data mode, provide the domain and datasetId. The actor calls the SODA endpoint ({domain}/resource/{datasetId}.json) with your SoQL filters ($where, $select, $order) applied server-side. Pagination is handled internally so you can pull up to 100,000 rows in one run. Every row is tagged with _domain, _dataset_id, and _scraped_at.

🧾 Input configuration

Discovery mode:

{
  "searchQuery": "restaurant inspections",
  "maxResults": 25
}

Data mode:

{
  "domain": "data.cdc.gov",
  "datasetId": "9bhg-hcku",
  "select": "state,sex,age_group,covid_19_deaths",
  "where": "state='United States'",
  "order": "covid_19_deaths DESC",
  "maxResults": 5000
}

📤 Output format

Discovery mode returns one record per matching dataset:

{
  "_type": "dataset",
  "domain": "data.cityofnewyork.us",
  "dataset_id": "4dx7-axux",
  "name": "Open Restaurants Inspections",
  "description": "The Open Restaurants Inspections dataset contains records of site setup inspections performed throughout the five boroughs of New York City.",
  "dataset_type": "dataset",
  "updated_at": "2026-04-22T12:50:29.000Z",
  "permalink": "https://data.cityofnewyork.us/d/4dx7-axux",
  "link": "https://data.cityofnewyork.us/Transportation/Open-Restaurants-Inspections/4dx7-axux",
  "scraped_at": "2026-07-15T04:18:03.185Z"
}

Captured from a live discovery run on restaurant inspections, first result returned.

Data mode returns each row as the dataset defines it, plus these tag fields:

Field	Description
🌐 _domain	Socrata portal domain (e.g. data.cdc.gov)
🆔 _dataset_id	Dataset 4x4 identifier
🕒 _scraped_at	ISO timestamp of capture
📊 ...	All native dataset columns as returned by SODA

💼 Common use cases

Government data journalism Search hundreds of portals in one query, then pull the rows that back a story into a flat table. Filter server-side with SoQL to keep runs small and cheap.

Public-health research Pull CDC datasets on outcomes, surveillance, or exposures by geography and demographic slice. Combine with HHS or CMS datasets for a fuller pipeline.

Municipal analytics Query NYC or LA operational data (311 requests, restaurant inspections, permits) for city-scale analysis. Automate weekly pulls to feed BI dashboards.

AI grounding Discover a dataset by keyword, pull a filtered slice, and feed it as grounded context to a Claude or ChatGPT agent. Combine with the domain-level indexer for a full open-data RAG layer.

🚀 Getting started

1. Open the actor and decide: discover a dataset or pull rows from one you already know.
2. For discovery, enter a Search datasets keyword (e.g. covid deaths) and leave domain and dataset ID blank.
3. For data mode, enter the Portal domain and Dataset ID (from discovery or from the dataset URL).
4. Optionally add SoQL filters: where (row filter), select (columns), order (sort).
5. Click Save & Start, then download as JSON, CSV, or Excel, or pull via API or MCP.

FAQ

Do I need an API key? No. Socrata's APIs are keyless. A free Socrata app token only raises rate limits for very large pulls; leave it blank for most runs.

Which portals are supported? Any Socrata-powered portal: hundreds of US federal, state, and city government sites, plus international portals that run on Socrata.

How do I find a dataset ID? Use discovery mode (search by keyword), or copy the 4x4 identifier from the dataset's URL (e.g. 9bhg-hcku).

Can I filter server-side? Yes. Full SoQL support: $where for row filters, $select for columns, $order for sorting. Filtering happens on Socrata's side, so you only pull (and pay for) the rows you want.

How am I charged? Pay per record at $0.001 per record ($1 per 1,000). The first 10 records are free for life on every Apify account. Empty runs are never charged.

Use in Claude, ChatGPT & any MCP agent

https://mcp.apify.com/?tools=themineworks/socrata-open-data

Or call it programmatically with the Apify client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/socrata-open-data').call({
  domain: 'data.cdc.gov',
  datasetId: '9bhg-hcku',
  where: "state='New York'",
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

🛠️ Complete your open data pipeline

Socrata is one portal family. Add the others:

• World Bank Indicators: GDP, inflation, trade, and 1,400+ development series by country.
• CMS Hospital Quality: star ratings and directory for 4,500+ US hospitals.
• openFDA Scraper: FDA drug, device, and food data through one unified endpoint.

Typical flow: discover a Socrata dataset by keyword, pull filtered rows, and cross-reference with hospital, drug, or macro data from the same suite.

Found a bug or have a feature request? Open an issue on the actor's Apify Console page or reach out through the Apify profile.