🌉 San Francisco Open Data Scraper

🚀 Export any San Francisco Open Data dataset in seconds. Tap 659 published datasets including business registrations, restaurants, building permits, 311 cases, parking citations, evictions, police incidents, and more, via the official Socrata SODA API. No API key, no registration.

The San Francisco Open Data Scraper is a universal export tool for every dataset on data.sfgov.org. The City and County of San Francisco publishes 659 datasets covering city operations, public safety, transportation, economy, environment, health, and culture. This Actor lets you pull any of them by passing the Socrata 4x4 dataset ID, optionally adding SoQL filters ($where, $select, $order, $q), and downloading the result as CSV, Excel, JSON, or XML.

The catalog spans every major SF civic data set, including building permits (i98e-djp9), registered businesses (pyih-qa8i), 311 service requests (vw6y-z8j6), parking citations (5cei-gny5), eviction notices (tu7p-pa2g), mobile food permits (rqzj-sfat), police incident reports (wg3w-h783), restaurant inspections, film locations, and historical crime statistics. Output preserves the dataset's native schema and appends three metadata fields: _datasetId, _datasetUrl, and _scrapedAt.

🎯 Target Audience	💡 Primary Use Cases
Civic researchers, journalists, prop-tech startups, GIS engineers, data scientists, public health analysts, real-estate firms, urban planners, students	Civic dashboards, FOIA-style export, permit/business/restaurant feeds, eviction and 311 monitoring, journalism investigations, ML training data on municipal events

📋 What the SF Open Data Scraper does

Four filtering knobs map straight to Socrata SoQL:

• 🆔 Dataset selector. Pick any of 659 datasets by 4x4 ID. Find IDs in the URL of any dataset page on data.sfgov.org.
• 🔍 WHERE clause. Standard SoQL $where, e.g. permit_type=3 AND filed_date>'2024-01-01'.
• 📋 SELECT clause. Limit returned columns via $select.
• 📈 ORDER clause. Sort with $order, e.g. filed_date DESC.
• 🔎 Full-text search. Free-text $q across all string columns.

Each record returns the dataset's native columns verbatim (with Socrata's internal :@computed_region_* lookup columns stripped to keep the output clean), plus three appended metadata fields: _datasetId, _datasetUrl, and _scrapedAt. Pagination is automatic and capped at 1,000,000 rows.

💡 Why it matters: San Francisco publishes one of the richest open-data catalogs of any U.S. city, but the SODA API has its own query language, paging quirks, and computed-region noise. This Actor turns that into a clean, paginated export with no Socrata code on your side.

📊 Data fields

Each record includes: _datasetId, _datasetUrl, _scrapedAt, agent_address, data_as_of, data_loaded_at, firm_address, firm_city, firm_name, firm_state, firm_zipcode, first_name, from_date, id, is_applicant, last_name, license1, permit_number, pts_agent_id, role, sf_business_license_number. All 21 field names come from a real production run, so what you see here is what lands in your dataset.

🚀 How to use

1. 📝 Sign up. Create a free account with $5 credit (takes 2 minutes).
2. 🌐 Open the Actor. Go to the San Francisco Open Data Scraper page on the Apify Store.
3. 🎯 Pick a dataset. Find the 4x4 ID on data.sfgov.org (it's in every dataset URL) and paste it in.
4. 🔍 Add optional filters. Type a SoQL $where, $order, $select, or full-text $q if you want a slice.
5. 🚀 Run it. Click Start and let the Actor collect your data.
6. 📥 Download. Grab your results in the Dataset tab as CSV, Excel, JSON, or XML.

⏱️ Total time from signup to downloaded dataset: 3-5 minutes. No coding required.

🔗 Recommended Actors

• 🌴 Los Angeles Open Data Scraper - Same Socrata pattern, every LA dataset
• 🏗️ Seattle Building Permits Scraper - Seattle DCI permits
• 🏠 Greatschools Scraper - U.S. K-12 school ratings
• 🏛️ James Edition Real Estate Scraper - Luxury international real estate
• 📰 PR Newswire Scraper - Press releases and corporate announcements

💡 Pro Tip: browse the complete ParseForge collection for more public-data scrapers.

⚠️ Disclaimer: this Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the City and County of San Francisco or Tyler Technologies / Socrata. All trademarks mentioned are the property of their respective owners. Only publicly available open data is collected.

🆘 Need Help?

If you hit a bug, have questions about setup, or need a scraper we haven't built yet, open our contact form or write to parseforge@protonmail.com. We also take on paid custom data projects.

For faster answers, join our Discord. It's the best place to get support and suggest new actors.