🇺🇸 Data.gov Scraper

🚀 Collect open dataset metadata from Data.gov in minutes. Search by keyword, topic, or category. Export dataset titles, descriptions, download links, and agency info. No coding, no account required.

🕒 Last updated: 2026-04-16 · 📊 20+ fields per dataset · 🔍 3 search filters · 📂 Topic + category · 🚫 No auth required

The Data.gov Scraper collects open data catalog metadata from the U.S. government's data portal, returning 20+ fields per dataset: title, description, agency, topic, format, download URL, update frequency, and license. Data.gov hosts over 300,000 datasets from 100+ federal agencies.

The Actor supports keyword search with topic and category filters, plus direct URL scraping.

🎯 Target Audience	💡 Primary Use Cases
Data scientists, policy researchers, journalists, civic tech teams, academic researchers, government analysts	Open data discovery, policy research, data journalism, civic technology, federal data monitoring

---

📋 What the Data.gov Scraper does

Three search filters:

• 🔍 Keyword search. Free-text search across dataset titles and descriptions.
• 📂 Topic filter. Subject areas (health, education, climate, finance, etc.).
• 📂 Category filter. Detailed categorization within topics.
• 🔗 URL mode. Paste a direct Data.gov search URL.

Each dataset record includes title, description, agency, topic, categories, data format, download URLs, update frequency, license, and portal URL.

💡 Why it matters: browsing Data.gov for relevant datasets means scrolling through 300,000+ listings. This Actor exports structured catalog metadata at scale for your open data pipelines or research projects.

---

🎬 Full Demo

🚧 Coming soon: a 3-minute walkthrough showing how to go from sign-up to a downloaded dataset.

---

⚙️ Input

Input	Type	Default	Behavior
startUrl	string	""	Direct Data.gov search URL.
searchQuery	string	""	Keyword search.
topics	array	[]	Subject area filters.
topicCategories	array	[]	Detailed categories within topics.

Example: climate datasets.

{
    "searchQuery": "climate change",
    "topics": ["Climate"],
    "maxItems": 50
}

Example: health data from CDC.

{
    "searchQuery": "CDC",
    "topics": ["Health"],
    "maxItems": 100
}

⚠️ Good to Know: Data.gov is the U.S. government's official open data portal. Datasets link to various formats (CSV, JSON, XML, API) hosted by individual agencies.

---

📊 Output

Each dataset record contains 20+ fields. Download the catalog as CSV, Excel, JSON, or XML.

🧾 Schema

Field	Type	Example
📝 title	string	"Daily Climate Normals"
📄 description	string	"NOAA climate normals for U.S. stations..."
🏛️ agency	string	"NOAA"
📂 topic	string	"Climate"
📂 categories	array	["Weather", "Environment"]
📦 format	string	"CSV"
🔗 downloadUrl	string	"https://catalog.data.gov/dataset/..."
📅 lastUpdated	string	"2026-01-15"
🔄 updateFrequency	string	"Annual"
📜 license	string	"Public Domain"
🔗 portalUrl	string	"https://catalog.data.gov/dataset/..."
🕒 scrapedAt	ISO 8601	"2026-04-16T00:00:00.000Z"

📦 Sample records

---

✨ Why choose this Actor

	Capability
🇺🇸	300,000+ datasets. Full U.S. federal open data catalog.
🔍	3 search filters. Keyword, topic, category.
📦	Format and download links. Direct URLs to CSV, JSON, XML, API.
🏛️	Agency data. Federal agency per dataset.
📅	Update frequency. Daily, monthly, annual cadence.
⚡	Scalable. Quick lookups to full catalog sweeps.
🚫	No authentication. Public open data portal.

📊 Data.gov hosts 300,000+ datasets from 100+ federal agencies. Structured access powers every open data project, policy research, and civic technology workflow.

---

📈 How it compares to alternatives

Approach	Cost	Coverage	Refresh	Filters	Setup
⭐ Data.gov Scraper (this Actor)	$5 free credit, then pay-per-use	Full catalog	Live per run	keyword, topic, category	⚡ 2 min
Manual Data.gov browsing	Free	One at a time	Manual	UI only	🕒 Hours
CKAN API (direct)	Free	Full	Real-time	Many	⏳ Hours

Pick this Actor when you want U.S. federal open data catalog metadata on demand, with topic and category filters.

---

🚀 How to use

1. 📝 Sign up. Create a free account with $5 credit (takes 2 minutes).
2. 🌐 Open the Actor. Go to the Data.gov Scraper page on the Apify Store.
3. 🎯 Set input. Enter a keyword, pick topics and categories.
4. 🚀 Run it. Click Start.
5. 📥 Download. Grab results in the Dataset tab.

⏱️ Total time: 3-5 minutes. No coding required.

---

💼 Business use cases

📊 Data Science & Research Discover federal datasets for research Build open data catalogs Track new publications by agency Analyze data coverage by topic	🏛️ Policy & Civic Tech Monitor federal data releases Build transparency dashboards Track agency publishing rates Power civic apps with open data
📰 Data Journalism Find datasets for investigative reporting Track data freshness across agencies Build story pipelines from government data Monitor new dataset publications	🏢 Business Intelligence Enrich models with federal data Track economic indicators by agency Build market data pipelines Monitor regulatory data releases

---

🔌 Automating Data.gov Scraper

• 🟢 Node.js. Install the apify-client NPM package.
• 🐍 Python. Use the apify-client PyPI package.
• 📚 See the Apify API documentation for full details.

The Apify Schedules feature lets you trigger this Actor on any cron interval. Weekly pulls catch new dataset publications.

---

❓ Frequently Asked Questions

---

🔌 Integrate with any app

• Make - Automate workflows
• Zapier - Connect 5,000+ apps
• Slack - Get notifications
• Airbyte - Data pipelines
• GitHub - Trigger from commits
• Google Drive - Export to Sheets

---

🔗 Recommended Actors

• 🇬🇧 Data.gov.uk Scraper - UK open data catalog
• 📊 USAspending Scraper - Federal spending data
• 📊 FRED Scraper - Economic data
• 📋 GSA eLibrary Scraper - Government contracts
• 📊 Indexmundi Scraper - Global indicators

💡 Pro Tip: browse the complete ParseForge collection for more government and open data scrapers.

---

🆘 Need Help? Open our contact form to request a new scraper, propose a custom data project, or report an issue.

---

⚠️ Disclaimer: this Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the U.S. government or Data.gov. All trademarks mentioned are the property of their respective owners. Only publicly available open data catalog metadata is collected.