India Statistics Scraper — MoSPI Data

Retrieve official Indian statistics through the Ministry of Statistics and Programme Implementation (MoSPI) eSankhyiki MCP service and export the results through Apify.

This is an independent integration, not an official MoSPI or Government of India product. The upstream statistics service is publicly accessible without an API key. Any Actor fee pays for the integration and operation; it does not buy exclusive access to government data.

What you can do

• Discover the live dataset catalog, including national accounts, consumer prices, industrial production, employment, education, health, energy and other subjects when available upstream.
• Inspect indicator definitions and valid filters before retrieving data.
• Fetch bounded, sequential pages of statistical observations.
• Export JSON, CSV or Excel using Apify dataset exports. JSON preserves nested record objects best; select record.<field> columns when exporting tabular formats.
• Keep source labels, units, years, zero values and nulls unchanged. Original responses and metadata are saved separately for verification.

Coverage and freshness vary by dataset. Availability in the catalog does not mean every dataset/filter combination has been tested or has recent data. This Actor retrieves structured statistics; it does not crawl every page, PDF, census microdata file or publication on the MoSPI website.

Quick start

Run with the default input:

{"action":"list_datasets"}

Find a dataset code in Results. Then use these actions in order:

1. get_indicators: choose an indicator and its frequency/base-year selector.
2. get_metadata: retrieve the valid geographic, time and category filters.
3. get_data: copy the exact selectors and filters into the data query.

get_data also repeats dataset, indicator and metadata discovery internally to capture the current definitions. It validates the live tool schema and published filter parameter contracts. It does not interpret a natural-language question or choose statistical concepts on your behalf. Metadata codes are dataset-specific; never reuse a code from another dataset.

Working example: PLFS annual unemployment

The repository includes examples/01-list-datasets.json through examples/04-plfs-unemployment.json. For metadata:

{
  "action": "get_metadata",
  "dataset": "PLFS",
  "indicatorParameters": {"frequency_code": 1},
  "metadataParameters": {"indicator_code": 3, "frequency_code": 1}
}

A bounded sample query using the discovered PLFS indicator:

{
  "action": "get_data",
  "dataset": "PLFS",
  "indicatorParameters": {
    "frequency_code": 1
  },
  "metadataParameters": {
    "indicator_code": 3,
    "frequency_code": 1
  },
  "filters": {
    "indicator_code": "3",
    "frequency_code": "1",
    "Format": "JSON"
  },
  "pageSize": 5,
  "maxItems": 10,
  "maxPages": 2
}

This fetches up to ten annual unemployment observations across the available periods and demographic categories. It is a connectivity and export example, not a single national unemployment estimate. Every record retains its year, state, gender, age group and other dimensions. Use metadata to construct a narrower analytical query; valid individual filter values do not guarantee that their combination has data. This sample deliberately reaches a limit and reports truncated: true.

For GDP, use examples/05-nas-gdp.json. It requests annual GDP under the 2022–23 base, preserving current/constant prices and provisional/revised status.

Input reference

Field	Purpose	Default
action	list_datasets, get_indicators, get_metadata, get_data, list_tools	list_datasets
dataset	Exact code from the live catalog	Required for indicator/metadata/data actions
indicatorParameters	Arguments for indicator discovery, excluding dataset	{}
metadataParameters	Arguments for metadata discovery, excluding dataset	{}
filters	Data filter keys and values from metadata	Required for get_data
pageSize	Requested observations per page, 1–1,000	100
maxItems	Maximum output observations, 1–100,000	1,000
maxPages	Maximum data pages, 1–1,000	10
startPage	First upstream page	1

Use list_tools to inspect the current MCP argument schemas. Put pagination in the Actor fields, not in filters. Request JSON from upstream; use Apify for CSV/Excel conversion. The upstream may return fewer rows than requested.

Output

Each dataset item has the following structure:

{
  "kind": "observation",
  "dataset": "PLFS",
  "page": 1,
  "sourceUrl": "https://mcp.mospi.gov.in/",
  "fetchedAt": "<UTC retrieval timestamp>",
  "record": {"<official field>": "<unchanged official value>"}
}

This illustrates the envelope, not a fabricated statistical observation. Discovery actions produce dataset, indicators, metadata or tool items. One observation represents one returned source row, which may contain several dimension fields.

The default key-value store contains:

• OUTPUT: row count, pages fetched, stop reason and truncated flag.
• TOOLS, DATASETS, INDICATORS, METADATA: discovery results as applicable.
• PAGE-00001, etc.: parsed source data pages; corresponding -RAW keys preserve the MCP response.
• ERROR: diagnostic information if execution fails. Earlier pages may already be stored; a failed run is not a complete dataset.

The Actor stops on an empty page, an output/page cap or a charging budget limit. A short page alone is not treated as proof of exhaustion. Repeated pages trigger an error to prevent endless duplicate output. Reaching a cap sets truncated: true, conservatively, even if the source happens to end at that same boundary. Pagination is not a snapshot transaction: source revisions during a run may change ordering.

Pricing and resources

The Actor supports automatic dataset-item billing and optional custom observation billing. When automatic dataset billing is configured, every saved dataset row (including discovery and metadata rows) is billed by the platform; the Actor does not add a custom result charge. Otherwise, if a custom result event is configured, it is charged only for saved statistical observations. The SDK respects the run's maximum-charge budget, and OUTPUT reports any resulting truncation. Prices and any additional platform usage charges on the Pricing tab are authoritative. Core features are available regardless of the user's Apify subscription tier.

No MoSPI credentials, browser, proxy or external VPS are required. Start with 256 MB memory and a 600-second run timeout for small queries. Larger queries may need more time. Requests run sequentially with a delay and bounded retries for temporary network errors, HTTP 429 and selected HTTP 5xx responses.

Troubleshooting

• Unknown dataset/filter: use discovery again; upstream schemas and catalogs can change.
• No observations: check the exact combination of year, base, frequency and dimensions. Missing data is not zero.
• HTTP 429 / timeout / 5xx: retry later or narrow the selection. The Actor does not bypass upstream controls.
• Unrecognized response shape: inspect the saved raw page and report the run ID. It is not silently converted into a successful empty result.
• Partial output: inspect OUTPUT.truncated, OUTPUT.stopReason, and the run status before analysis.

Developers

Python 3.12:

pip install -r requirements.txt
python -m unittest discover -s tests -v
mkdir -p storage/key_value_stores/default
cp examples/04-plfs-unemployment.json storage/key_value_stores/default/INPUT.json
python -m src

Apify reads .actor/actor.json, the Dockerfile, README, input schema, output schema and dataset view from this repository. This is a run-and-finish Actor, not a permanent MCP server. It can be invoked by AI clients through Apify's Actor integration.

Owner deployment instructions and copy-ready Store fields: docs/STORE-LISTING-ZH.md. Validation details: docs/VERIFICATION.md.

Source and attribution

Source: MoSPI eSankhyiki MCP. Cite MoSPI and the dataset, indicator, period, units and retrieval date when using the statistics. Upstream data remains subject to its applicable attribution and reuse conditions. The code license does not grant rights in third-party data. This integration does not certify the accuracy, completeness or timeliness of the source.