Socrata Open Data Scraper: export any city or state SODA dataset

Exports rows from any Socrata open-data portal (NYC, Chicago, Washington, San Francisco and hundreds more) without an API key. Give it a portal domain and a dataset ID, optionally add SoQL filters, and get clean JSON/CSV/Excel rows. It can also list every dataset on a portal, or return a dataset's column schema. Built for analysts, journalists, civic-tech developers and AI agents that need public-sector data. Price: $0.0005 per returned record (about $0.50 per 1,000; lower on paid Apify plans).

Running it with an empty input {} returns 1,000 recent NYC 311 service requests (about $0.50 at the Free-plan price). Set maxRecords first if you want to spend less.

Example input

{ "mode": "rows", "domain": "data.cityofnewyork.us", "datasetId": "erm2-nwe9",
  "where": "complaint_type = 'Noise - Residential'", "order": "created_date DESC", "maxRecords": 100 }

Example output (real run, mode rows)

{
  "_domain": "data.cityofnewyork.us",
  "_datasetId": "erm2-nwe9",
  "_mode": "rows",
  "data": {
    "unique_key": "70691427",
    "created_date": "2026-10-09T02:56:50.000",
    "agency": "DOT",
    "complaint_type": "Street Condition",
    "descriptor": "Pothole",
    "incident_zip": "11415",
    "status": "Open",
    "borough": "QUEENS"
  }
}

(Trimmed: data holds every column the portal returns for that row.)

Real catalog output (one record per dataset): {"_domain":"data.cityofchicago.org","_mode":"catalog","datasetId":"ydr8-5enu","name":"Building Permits","type":"dataset","category":"Buildings","updatedAt":"2026-10-10T12:06:20.000Z","permalink":"https://data.cityofchicago.org/d/ydr8-5enu","link":"https://data.cityofchicago.org/Buildings/Building-Permits/ydr8-5enu"}

Modes

mode	Returns	Needs
rows (default)	One record per dataset row; row columns are inside data	domain, datasetId
catalog	One record per dataset on the portal	domain (optional q, category)
metadata	One record with the dataset's columns (name, fieldName, type)	domain, datasetId

Input

Field	Description
mode	rows, catalog or metadata. Default rows.
domain	Portal host, e.g. data.cityofnewyork.us. Default data.cityofnewyork.us.
datasetId	The xxxx-xxxx code in the dataset URL, e.g. erm2-nwe9. Default erm2-nwe9.
select, where, order	Optional SoQL clauses (rows mode).
q	Full-text search (rows) or keyword filter (catalog).
category	Dataset category filter (catalog mode).
maxRecords	Stop after N records; default 1000; 0 = no limit (can be millions of rows, set a cap first).
pageSize	Rows per request, 1-50000; default 1000.
appToken	Optional Socrata app token; never required.

Output fields

• All modes: _domain, _mode.
• rows: _datasetId, data (object with the dataset's own columns).
• catalog: datasetId, name, description, type, category, updatedAt, createdAt, permalink, link, pageViewsTotal (when the portal reports it). Note: the catalog rowCount field is derived from the portal's column list, so treat it as a column count, not a row count.
• metadata: datasetId, name, description, category, attribution, createdAt, rowsUpdatedAt, viewCount, downloadCount, columnCount, columns[].

Pricing

Pay per event: one dataset-item event of $0.0005 per record returned (Free plan; lower on Bronze and above), whether it is a row, a catalog entry or a metadata record. No charge for runs that return nothing.

Limits

• Public datasets only; private or login-required datasets fail. Socrata (SODA2) portals only.
• A run that returns no records fails with exit code 91.
• Unauthenticated portals may rate-limit; the Actor retries with backoff. Add appToken or lower pageSize if it persists.
• Default memory is 256 MB (measured peak about 76 MB).

Maximum charge per run

If you set a maximum total charge, each page request asks the portal only for what the remaining budget and maxRecords can cover. When the budget runs out the run stops with the records already delivered and logs "Max total charge reached". Each record is written and billed in one step, and a restarted run resumes from the saved offset, so nothing is written or billed twice.

Use from an AI agent / API

Python:

from apify_client import ApifyClient
client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("datamule/socrata-soda-extractor").call(
    run_input={"domain": "data.cityofchicago.org", "datasetId": "ijzp-q8t2",
               "order": "date DESC", "maxRecords": 50})
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())

MCP (Claude, Cursor and other agents): add https://mcp.apify.com?tools=datamule/socrata-soda-extractor as a remote MCP server with your Apify token. The Actor then appears as a tool whose arguments are the Input fields above.

Agent tips: to find a dataset ID, run mode: "catalog" with q; to learn column names before writing where, run mode: "metadata"; always set maxRecords.

FAQ

Do I need a Socrata account or API key? No.

Where do I find the dataset ID? It is the xxxx-xxxx code at the end of the dataset URL, or use catalog mode.

Why are my rows nested under data? Every portal has different columns, so the row's own columns are kept together under data.

Can I get everything? Set maxRecords to 0. Large datasets cost proportionally ($0.50 per 1,000 rows).

Does it work on non-US portals? Any portal running the Socrata SODA API works; give its host as domain.