data.gov.in Scraper: Any Dataset ID, Up to 100k Rows

Pay only for results delivered. Browse all Actors.

💰 From $1.00 / 1,000 results.

How to pull any India government open dataset as clean JSON

Paste a resource ID from data.gov.in and this actor returns that dataset as flat JSON rows, up to 100,000 records per run, paginated automatically, with server-side field filters applied before anything is fetched. Foreign trade, mandi commodity prices, census, agriculture, health, energy, transport: if it is on data.gov.in, this pulls it.

It wraps India's official OGD API, so there is no anti-bot fragility, no headless browser, and no rate-limit war. Bring the free key you can register for in a minute, or start with the shared sample key.

✅ No login and no scraping fragility · ✅ Any of thousands of datasets · ✅ Server-side filters · ✅ Up to 100,000 rows per run · ✅ MCP-ready for AI agents

Pull a whole dataset by resource ID

The minimum viable run. resourceId is the only required input.

{
  "resourceId": "9ef84268-d588-465a-a308-a864a43d0070",
  "maxResults": 1000
}

To find a resource ID: open a dataset on data.gov.in, click its API tab, and copy the UUID from the URL.

Filter mandi prices to one state and commodity

Filters are field=value strings, translated into the OGD API's own filters[field] parameters, so the narrowing happens on the government's server, not after you have paid to download the rest.

{
  "resourceId": "9ef84268-d588-465a-a308-a864a43d0070",
  "filters": ["state=Maharashtra", "commodity=Onion"],
  "maxResults": 500
}

Field names must match the dataset's own column names exactly. Run once without filters to see the column names in the output, then add them.

Backfill a large dataset with your own free API key

The shared sample key is heavily rate-limited because everyone uses it. Register a free key on data.gov.in before any serious pull, it takes a minute and it is the difference between a run that completes and one that stalls.

{
  "resourceId": "9ef84268-d588-465a-a308-a864a43d0070",
  "apiKey": "your-free-data-gov-in-key",
  "maxResults": 100000
}

100,000 is the per-run ceiling. The actor pages through in blocks of 100 and stops when it reaches your limit or the dataset's own total.

Sample a dataset cheaply before committing

Set a small maxResults for a first look. The run log prints the dataset title and its total record count as soon as the first page returns, so you learn how big the dataset is before you pay to pull it.

{
  "resourceId": "3b01bcb8-0b14-4abf-b6f2-c1bfd384ba69",
  "maxResults": 25
}

What data do you get from data.gov.in

The output shape is the dataset's own shape. This is a passthrough: every column the government publishes for that resource comes through unchanged, so the field list depends entirely on which dataset you asked for. Two audit fields are appended to every row.

Real record from the daily mandi price dataset:

{
  "state": "Keralam",
  "district": "Idukki",
  "market": "Kattappana Market",
  "commodity": "Water Melon",
  "variety": "Other",
  "grade": "Medium",
  "arrival_date": "15/07/2026",
  "min_price": 2000,
  "max_price": 2600,
  "modal_price": 2300,
  "_resource_id": "9ef84268-d588-465a-a308-a864a43d0070",
  "_scraped_at": "2026-07-15T04:14:45.668Z"
}

Field	Description
(dataset columns)	Every field the source dataset publishes, unmodified
🆔 _resource_id	The resource ID this row came from, lets you merge several datasets into one table and still know the provenance
🕒 _scraped_at	ISO timestamp of capture

A final _type: "summary" row reports dataset_title, records, total_available, and charged_for, so you can always tell how much of a dataset you actually pulled.

How does this work with no anti-bot and no browser

data.gov.in publishes a real REST API. This actor calls it directly over plain HTTP, no browser, no proxy, no fingerprinting, no captcha. What it adds on top is the tedious part: an inconsistent response envelope normalised, offset pagination handled, field=value filter strings translated into the API's parameter format, and automatic retries with capped backoff on 429 and 5xx responses.

The API key can be yours (free, one-minute registration) or the shared public sample key. The actor uses the sample key by default and warns you in the log when it does, because that key is shared across every anonymous user and throttles hard.

What does this cost

Event	Price	You pay when
Record delivered	$0.002	A record lands in your dataset

$2.00 per 1,000 records. Empty results, bad resource IDs, and failed requests are never charged, you only pay for rows actually written.

Common use cases

Commodity and agriculture pricing. Track mandi prices by state, district, and commodity over time.

Trade analysis. Pull India's export and import figures by commodity code and partner country.

Public policy and research. Census, health, energy, and transport datasets as clean rows instead of PDFs.

LLM and RAG pipelines. Authoritative Indian government data in a flat, stable JSON shape with provenance on every row.

Getting started

1. Find your dataset on data.gov.in, open its API tab, and copy the resource UUID.
2. Paste it into resourceId.
3. Run once with a small maxResults to see the column names.
4. Add filters using those exact column names, and register a free apiKey before a large pull.
5. Click Start, then export as JSON, CSV, or Excel, or pull it via API or MCP.

Run it on a schedule

1. Run the actor once with the input you want repeated, then click Save as a task.
2. In the Apify Console, go to Schedules → Create new.
3. Name it, set your timezone, and pick a frequency or a cron expression (e.g. 0 6 * * *).
4. Under Actors or tasks to run, add the task you saved.
5. Save. Nothing is charged just for a schedule existing.

Full options are in Apify's Schedules documentation.

FAQ

Do I need an API key? Not to start, a shared public sample key is used by default. It is heavily rate-limited, so register your own free key on data.gov.in before any real pull.

How do I find a resource ID? Open the dataset on data.gov.in, click the API tab, and copy the UUID from the URL.

Which fields will I get? Whatever the dataset publishes. This is a passthrough, so the schema is the source's schema plus _resource_id and _scraped_at.

How do I know what to filter on? Run once with no filters and a small maxResults, read the column names in the output, then filter on those exact names.

How many records can I pull in one run? Up to 100,000, fetched 100 at a time.

What happens if the resource ID is wrong? The API returns an error, the actor logs it and stops, and nothing is charged.

Is this data free to use? data.gov.in publishes under the Government Open Data Licence, India. Check the licence terms on the specific dataset page for attribution and redistribution requirements.

Use from Claude, ChatGPT and any MCP agent

https://mcp.apify.com/?tools=themineworks/india-data-gov-scraper

Or call it programmatically:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/india-data-gov-scraper').call({
  resourceId: '9ef84268-d588-465a-a308-a864a43d0070',
  filters: ['state=Maharashtra'],
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Complete your India data pipeline

• JustDial Scraper: India business listings with phone, address, and geo-coordinates.
• IndiaMART Scraper: B2B supplier leads by category.
• Naukri Jobs Scraper: India job market data with salaries normalised to lakhs.

Typical flow: data.gov.in provides the macro picture, JustDial and IndiaMART supply the firm-level detail, Naukri shows where hiring is moving.

---

Disclaimer: This actor is an independent tool and is not affiliated with or endorsed by the Government of India. Data is retrieved from the official OGD platform; check each dataset's licence for attribution and redistribution terms.

Questions or need a custom field set? Reach out through the Apify profile.

Related guides

• India Government Data API: How to Pull Any data.gov.in Dataset Without the Documentation Confusion

Last verified: 2026-08