Socrata Open Data Scraper: SoQL Filters, No API Key

Pay only for results delivered. Browse all Actors.

💰 From $0.60 / 1,000 results.

Two modes in one actor: find datasets across every Socrata government portal, then pull filtered rows out of any of them with SoQL. No API key.

Why use this Socrata open data scraper

Hundreds of US government bodies publish open data on Socrata, from the CDC and HHS to New York City, Texas and countless county portals. They all share one query language and one URL shape, and almost nobody knows it.

This actor gives you both halves of the workflow:

Discovery mode. Give a searchQuery and it searches the Socrata catalog across every portal at once, returning the dataset name, description, row count, portal domain, dataset id and permalink. This is how you find out that the data you need already exists and where it lives.

Data mode. Give a domain and a datasetId and it pulls actual rows, with full SoQL support: $where for filtering, $select for projection and $order for sorting, applied server-side so you download only what you asked for.

No API key. Socrata's app token is optional and only raises rate limits; pass one in appToken if you have it and you are pulling large volumes.

Data-mode rows are passed through exactly as the portal publishes them, plus _domain, _dataset_id and _scraped_at so you always know a row's provenance.

Find datasets on a topic across every Socrata portal

Discovery mode. Give a search query with no domain and dataset id and the actor searches the whole Socrata catalog.

{
  "searchQuery": "restaurant inspections",
  "maxResults": 100
}

Pull rows from a known dataset

Data mode. domain is the portal host and datasetId is the four-by-four id from the dataset's URL, for example 43nn-pn8j.

{
  "domain": "data.cityofnewyork.us",
  "datasetId": "43nn-pn8j",
  "maxResults": 5000
}

Filter rows server-side with SoQL

$where runs on the portal, not on your machine, so you download only the matching rows. SoQL supports comparison, AND/OR, LIKE, within_circle and date literals.

{
  "domain": "data.cityofnewyork.us",
  "datasetId": "43nn-pn8j",
  "where": "boro='Manhattan' AND score > 20",
  "order": "inspection_date DESC",
  "maxResults": 2000
}

Return only the columns you need

$select cuts the payload to specific columns, which matters when a dataset has 60 of them and you want 4.

{
  "domain": "data.cdc.gov",
  "datasetId": "9mfq-cb36",
  "select": "submission_date, state, tot_cases, new_case",
  "where": "state='CA'",
  "order": "submission_date DESC",
  "maxResults": 1000
}

What data you get back

The shape depends on which mode you ran.

Discovery mode returns one record per dataset:

Field	What it is
_type	Always dataset
domain	Portal host the dataset lives on
dataset_id	The four-by-four dataset identifier
name	Dataset title
description	Dataset description
dataset_type	Socrata asset type
rows_count	Row count as the portal reports it
updated_at	Last update timestamp
permalink	Stable link to the dataset
link	Portal link to the dataset
scraped_at	ISO timestamp of capture

Data mode returns the portal's own row, unchanged, plus three provenance fields: _domain, _dataset_id and _scraped_at. Column names and types are whatever that dataset publishes, so they differ from portal to portal.

A final _type: "summary" record reports which mode ran, the record count and the charged count.

How it works

Discovery mode calls api.us.socrata.com/api/catalog/v1 with your query, paging 100 at a time.

Data mode calls https://<domain>/resource/<datasetId>.json with your SoQL parameters mapped to $where, $select, $order, $limit and $offset, paging 1,000 rows at a time to keep memory light.

Mode is chosen automatically: supply both domain and datasetId and you get data mode; supply only searchQuery and you get discovery.

An appToken is sent as X-App-Token when you provide one. It is optional and only raises your rate limit. 429 and 5xx responses are retried with capped backoff, and a bad domain, dataset id or SoQL expression surfaces the portal's own error message in the log rather than failing silently.

Who uses Socrata open data data

Public health and safety analysis. CDC and HHS datasets, filtered to your state and date range server-side.

Civic and local journalism. Restaurant inspections, 311 complaints, permits and crime data, all queryable through the same interface.

Location intelligence. Business licences and permits by geography, joined against your own data.

Dataset discovery. Before commissioning a data collection project, run discovery mode. The dataset frequently already exists.

What it costs

One pay-per-event charge, record-scraped, billed per record that actually lands in your dataset. Your rate depends on your own Apify discount tier:

Your Apify tier	Per record	Per 1,000
FREE	$0.002	$2.00
BRONZE	$0.0016	$1.60
SILVER	$0.00125	$1.25
GOLD / PLATINUM / DIAMOND	$0.001	$1.00

Nothing else is billed. Failed requests, blocked pages, empty searches and error records are never charged, so a run that returns no records costs nothing. There is no subscription and no monthly minimum.

Run it on a schedule

Scheduled runs are the pattern most people end up on: smaller, more frequent runs finish faster, keep the data fresh, and cost exactly the same per record as one big run.

1. On this actor's page, click ... -> Schedule Actor (or Console -> Schedules -> Create new).
2. Pick a frequency. @daily suits most standing queries.
3. Your saved input is reused on every run and each run appends to the dataset.
4. Wire the dataset to Google Sheets, Slack, or a webhook from the Integrations tab so new rows reach you without you looking.

FAQ

Do I need a Socrata API key? No. The appToken is optional and only raises rate limits for high-volume pulls.

How do I find a dataset id? Run discovery mode first, or take it from the dataset's URL. It is the four-by-four identifier like 43nn-pn8j.

What is SoQL? Socrata's SQL-like query language. $where filters, $select projects, $order sorts, and all of it runs on the portal so you transfer only what you need.

Which portals are covered? Discovery runs against Socrata's US catalog service, which indexes portals across the platform. Data mode works against any Socrata domain you name, whether or not it appeared in discovery.

Why do my columns differ between datasets? Because data mode passes the portal's row through unchanged. Every dataset publishes its own schema. The _domain and _dataset_id fields tell you which one a row came from.

What happens if my SoQL is wrong? The portal's error response is surfaced in the log with a pointer to check the domain, dataset id and SoQL, and the run stops without charging for rows it never delivered.

Can I run discovery and data mode in one run? No. Supplying domain plus datasetId selects data mode and the searchQuery is ignored. Run them as two steps.

Use it from Claude, ChatGPT and any MCP agent

Hosted MCP endpoint, no install, OAuth on first connect:

https://mcp.apify.com/?tools=themineworks/socrata-open-data

Claude Desktop / Cursor config with token auth:

{
  "mcpServers": {
    "socrata": {
      "url": "https://mcp.apify.com/?tools=themineworks/socrata-open-data",
      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
    }
  }
}

Or call it from code with the Apify client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/socrata-open-data').call({
  "domain": "data.cityofnewyork.us",
  "datasetId": "43nn-pn8j",
  "where": "boro='Manhattan'",
  "maxResults": 1000
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Related actors

• India Data.gov Scraper: the equivalent for India's open government data.
• CMS Hospital Quality: US hospital quality measures.
• NPI Registry Scraper: US healthcare provider records.
• USAspending Scraper: federal award data.

---

Disclaimer: This actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Tyler Technologies or Socrata. Data is published by the individual government agencies that own each portal, under their own terms.

Found a bug or want a field added? Open an issue from the actor's Apify Console page or reach out through the The Mine Works profile.

Related guides

• Socrata API: How to Pull CDC, HHS, NYC, and 200+ Government Data Portals

Last verified: 2026-08