# India Statistics Scraper — MoSPI Data (`happyfirst/dataindia-apify-actor`) Actor

Extract Indian official statistics from MoSPI. Discover datasets, browse indicators and filters, and export GDP, prices, employment and other statistical data with source provenance.从印度统计与计划实施部 MoSPI 的官方 MCP 服务提取统计数据，支持数据集发现、指标和筛选条件查询、分页取数及结构化导出，保留数据来源与统计口径。目录当前列出 27 个数据集，实际覆盖以运行时官方返回为准。

- **URL**: https://apify.com/happyfirst/dataindia-apify-actor.md
- **Developed by:** [happyfirst](https://apify.com/happyfirst) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## India Statistics Scraper — MoSPI Data

Retrieve official Indian statistics through the Ministry of Statistics and Programme Implementation (MoSPI) eSankhyiki MCP service and export the results through Apify.

This is an independent integration, not an official MoSPI or Government of India product. The upstream statistics service is publicly accessible without an API key. Any Actor fee pays for the integration and operation; it does not buy exclusive access to government data.

### What you can do

- Discover the live dataset catalog, including national accounts, consumer prices, industrial production, employment, education, health, energy and other subjects when available upstream.
- Inspect indicator definitions and valid filters before retrieving data.
- Fetch bounded, sequential pages of statistical observations.
- Export JSON, CSV or Excel using Apify dataset exports. JSON preserves nested `record` objects best; select `record.<field>` columns when exporting tabular formats.
- Keep source labels, units, years, zero values and nulls unchanged. Original responses and metadata are saved separately for verification.

Coverage and freshness vary by dataset. Availability in the catalog does not mean every dataset/filter combination has been tested or has recent data. This Actor retrieves structured statistics; it does not crawl every page, PDF, census microdata file or publication on the MoSPI website.

### Quick start

Run with the default input:

```json
{"action":"list_datasets"}
```

Find a dataset code in Results. Then use these actions in order:

1. `get_indicators`: choose an indicator and its frequency/base-year selector.
2. `get_metadata`: retrieve the valid geographic, time and category filters.
3. `get_data`: copy the exact selectors and filters into the data query.

`get_data` also repeats dataset, indicator and metadata discovery internally to capture the current definitions. It validates the live tool schema and published filter parameter contracts. It does not interpret a natural-language question or choose statistical concepts on your behalf. Metadata codes are dataset-specific; never reuse a code from another dataset.

### Working example: PLFS annual unemployment

The repository includes `examples/01-list-datasets.json` through `examples/04-plfs-unemployment.json`. For metadata:

```json
{
  "action": "get_metadata",
  "dataset": "PLFS",
  "indicatorParameters": {"frequency_code": 1},
  "metadataParameters": {"indicator_code": 3, "frequency_code": 1}
}
```

A bounded sample query using the discovered PLFS indicator:

```json
{
  "action": "get_data",
  "dataset": "PLFS",
  "indicatorParameters": {
    "frequency_code": 1
  },
  "metadataParameters": {
    "indicator_code": 3,
    "frequency_code": 1
  },
  "filters": {
    "indicator_code": "3",
    "frequency_code": "1",
    "Format": "JSON"
  },
  "pageSize": 5,
  "maxItems": 10,
  "maxPages": 2
}
```

This fetches up to ten annual unemployment observations across the available periods and demographic categories. It is a connectivity and export example, not a single national unemployment estimate. Every record retains its year, state, gender, age group and other dimensions. Use metadata to construct a narrower analytical query; valid individual filter values do not guarantee that their combination has data. This sample deliberately reaches a limit and reports `truncated: true`.

For GDP, use `examples/05-nas-gdp.json`. It requests annual GDP under the 2022–23 base, preserving current/constant prices and provisional/revised status.

### Input reference

| Field | Purpose | Default |
| --- | --- | --- |
| `action` | `list_datasets`, `get_indicators`, `get_metadata`, `get_data`, `list_tools` | `list_datasets` |
| `dataset` | Exact code from the live catalog | Required for indicator/metadata/data actions |
| `indicatorParameters` | Arguments for indicator discovery, excluding `dataset` | `{}` |
| `metadataParameters` | Arguments for metadata discovery, excluding `dataset` | `{}` |
| `filters` | Data filter keys and values from metadata | Required for `get_data` |
| `pageSize` | Requested observations per page, 1–1,000 | 100 |
| `maxItems` | Maximum output observations, 1–100,000 | 1,000 |
| `maxPages` | Maximum data pages, 1–1,000 | 10 |
| `startPage` | First upstream page | 1 |

Use `list_tools` to inspect the current MCP argument schemas. Put pagination in the Actor fields, not in `filters`. Request JSON from upstream; use Apify for CSV/Excel conversion. The upstream may return fewer rows than requested.

### Output

Each dataset item has the following structure:

```json
{
  "kind": "observation",
  "dataset": "PLFS",
  "page": 1,
  "sourceUrl": "https://mcp.mospi.gov.in/",
  "fetchedAt": "<UTC retrieval timestamp>",
  "record": {"<official field>": "<unchanged official value>"}
}
```

This illustrates the envelope, not a fabricated statistical observation. Discovery actions produce `dataset`, `indicators`, `metadata` or `tool` items. One `observation` represents one returned source row, which may contain several dimension fields.

The default key-value store contains:

- `OUTPUT`: row count, pages fetched, stop reason and `truncated` flag.
- `TOOLS`, `DATASETS`, `INDICATORS`, `METADATA`: discovery results as applicable.
- `PAGE-00001`, etc.: parsed source data pages; corresponding `-RAW` keys preserve the MCP response.
- `ERROR`: diagnostic information if execution fails. Earlier pages may already be stored; a failed run is not a complete dataset.

The Actor stops on an empty page, an output/page cap or a charging budget limit. A short page alone is not treated as proof of exhaustion. Repeated pages trigger an error to prevent endless duplicate output. Reaching a cap sets `truncated: true`, conservatively, even if the source happens to end at that same boundary. Pagination is not a snapshot transaction: source revisions during a run may change ordering.

### Pricing and resources

The Actor supports automatic dataset-item billing and optional custom observation billing. When automatic dataset billing is configured, every saved dataset row (including discovery and metadata rows) is billed by the platform; the Actor does not add a custom result charge. Otherwise, if a custom `result` event is configured, it is charged only for saved statistical observations. The SDK respects the run's maximum-charge budget, and OUTPUT reports any resulting truncation. Prices and any additional platform usage charges on the Pricing tab are authoritative. Core features are available regardless of the user's Apify subscription tier.

No MoSPI credentials, browser, proxy or external VPS are required. Start with 256 MB memory and a 600-second run timeout for small queries. Larger queries may need more time. Requests run sequentially with a delay and bounded retries for temporary network errors, HTTP 429 and selected HTTP 5xx responses.

### Troubleshooting

- **Unknown dataset/filter:** use discovery again; upstream schemas and catalogs can change.
- **No observations:** check the exact combination of year, base, frequency and dimensions. Missing data is not zero.
- **HTTP 429 / timeout / 5xx:** retry later or narrow the selection. The Actor does not bypass upstream controls.
- **Unrecognized response shape:** inspect the saved raw page and report the run ID. It is not silently converted into a successful empty result.
- **Partial output:** inspect `OUTPUT.truncated`, `OUTPUT.stopReason`, and the run status before analysis.

### Developers

Python 3.12:

```bash
pip install -r requirements.txt
python -m unittest discover -s tests -v
mkdir -p storage/key_value_stores/default
cp examples/04-plfs-unemployment.json storage/key_value_stores/default/INPUT.json
python -m src
```

Apify reads `.actor/actor.json`, the Dockerfile, README, input schema, output schema and dataset view from this repository. This is a run-and-finish Actor, not a permanent MCP server. It can be invoked by AI clients through Apify's Actor integration.

Owner deployment instructions and copy-ready Store fields: [中文上架指南](docs/STORE-LISTING-ZH.md). Validation details: [verification](docs/VERIFICATION.md).

### Source and attribution

Source: [MoSPI eSankhyiki MCP](https://mcp.mospi.gov.in/). Cite MoSPI and the dataset, indicator, period, units and retrieval date when using the statistics. Upstream data remains subject to its applicable attribution and reuse conditions. The code license does not grant rights in third-party data. This integration does not certify the accuracy, completeness or timeliness of the source.

# Actor input Schema

## `action` (type: `string`):

Discover datasets first, then indicators, metadata and data.

## `dataset` (type: `string`):

Exact code from List datasets, e.g. PLFS or NAS.

## `indicatorParameters` (type: `object`):

Optional get\_indicators arguments from List tools, excluding dataset. For example {"frequency\_code":1}.

## `metadataParameters` (type: `object`):

Exact get\_metadata arguments excluding dataset, e.g. indicator\_code and frequency\_code obtained from discovery.

## `filters` (type: `object`):

Exact filters obtained from Get filter options. Include indicator/base/frequency selectors as required. Use pagination fields below for page/limit.

## `pageSize` (type: `integer`):

Rows requested per page. Reaching a cap is reported in OUTPUT; it does not imply complete coverage.

## `maxItems` (type: `integer`):

Maximum output observations. Reaching a cap is reported in OUTPUT; it does not imply complete coverage.

## `maxPages` (type: `integer`):

Maximum data pages. Reaching a cap is reported in OUTPUT; it does not imply complete coverage.

## `startPage` (type: `integer`):

Starting page. Reaching a cap is reported in OUTPUT; it does not imply complete coverage.

## Actor input object example

```json
{
  "action": "list_datasets",
  "indicatorParameters": {},
  "metadataParameters": {},
  "filters": {},
  "pageSize": 100,
  "maxItems": 1000,
  "maxPages": 10,
  "startPage": 1
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `rawResponses` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("happyfirst/dataindia-apify-actor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("happyfirst/dataindia-apify-actor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call happyfirst/dataindia-apify-actor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,happyfirst/dataindia-apify-actor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/A4tdgofr4GM5la2H0/builds/44x597BsMGogmfw29/openapi.json