# World Bank Scraper: 16,000 Development Indicators (`arman-bd/world-bank-indicators-scraper`) Actor

Scrape World Bank open data: any of 16,000+ indicators across 200+ countries and 60 years. GDP, inflation, population, energy, health and education series. No key.

- **URL**: https://apify.com/arman-bd/world-bank-indicators-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Business, Developer tools, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 indicator scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## World Bank Scraper: 16,000 Development Indicators

![World Bank Scraper: GDP, inflation, population and every other indicator, one flat row per country, indicator and year](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/world-bank-indicators-scraper.jpg)

**World Bank Scraper** exports any of the World Bank's 16,000+ development indicators, GDP, inflation, population, trade, energy, health, education, for 200+ countries and every aggregate, back to 1960, as one flat row per country-indicator-year.

It reads the public **World Bank Indicators API** directly: **no proxy setup, no browser, no credentials to manage.** A single request returns up to 20,000 observations, so a full indicator across every country and every year is usually one round trip.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/world-bank-indicators-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/world-bank-indicators-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `countryIso` | ISO-3 code (`USA`), falling back to ISO-2 on the handful of aggregate rows that lack one |
| `countryName` | Country or aggregate name as the World Bank publishes it |
| `indicatorCode`, `indicatorName` | e.g. `NY.GDP.MKTP.CD` and `GDP (current US$)` |
| `year` | Observation year, as a number |
| `value` | The observation, `null` where the World Bank has no data |
| `unit` | Unit string when the series carries one |
| `region` | World Bank region, joined from the country reference table, `Aggregates` for non-country rows |
| `incomeLevel` | Income classification, e.g. `Lower middle income` |
| `decimalPlaces` | Publisher's suggested display precision |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, and any indicator that failed.

### Common use cases

- **Macro research and country risk scoring.** Pull GDP, inflation and debt for a watchlist in one run.
- **Market sizing.** Population and GDP per capita by country, joined to your own segmentation.
- **Economics dashboards.** A scheduled refresh keeps a time-series store current without any key management.
- **Development and ESG analysis.** Energy, emissions, health and education series with region and income level already attached.
- **Data journalism.** Reproducible extracts with the indicator code recorded on every row.

### Quick start

GDP and population for three economies:

```json
{
 "countries": ["USA", "GBR", "DEU"],
 "indicators": ["NY.GDP.MKTP.CD", "SP.POP.TOTL"]
}
```

Every country, one indicator, this century:

```json
{
 "countries": ["all"],
 "indicators": ["FP.CPI.TOTL.ZG"],
 "fromYear": 2000,
 "toYear": 2024
}
```

A complete gap-marked grid for a single country, for time-series work that needs the missing years present:

```json
{
 "countries": ["BD"],
 "indicators": ["NY.GDP.PCAP.CD"],
 "skipNullValues": false
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `indicators` | array | - | **Required.** Indicator codes, or full `data.worldbank.org/indicator/.` URLs. |
| `countries` | array | `all` | ISO-2 or ISO-3 codes. Aggregates (`EMU`, `WLD`, `SAS`) work. Empty or `all` returns everything. |
| `fromYear` | integer | - | Earliest year. Most series start in 1960. |
| `toYear` | integer | - | Latest year. Defaults to the most recent published. |
| `skipNullValues` | boolean | `true` | Drop country-years with no observation. Turn off to keep the full grid. |
| `maxResults` | integer | `0` | Cap on total records. `0` = no limit. |

Setting only `fromYear` or only `toYear` is fine, the other end of the range is filled in with 1960 or the current year. Passing `all` alongside specific countries makes `all` win, because the source treats them as mutually exclusive.

### Finding an indicator code

Open the indicator on [data.worldbank.org](https://data.worldbank.org/indicator) and read the code out of the URL:

| URL you see | Code |
|---|---|
| `data.worldbank.org/indicator/NY.GDP.MKTP.CD` | `NY.GDP.MKTP.CD` |
| `data.worldbank.org/indicator/SP.POP.TOTL` | `SP.POP.TOTL` |
| `data.worldbank.org/indicator/FP.CPI.TOTL.ZG` | `FP.CPI.TOTL.ZG` |

You can paste the whole URL, the Actor extracts the code itself.

### Output example

```json
{
 "countryIso": "USA",
 "countryName": "United States",
 "indicatorCode": "NY.GDP.MKTP.CD",
 "indicatorName": "GDP (current US$)",
 "year": 2025,
 "value": 30769700000000,
 "unit": null,
 "region": "North America",
 "incomeLevel": "High income",
 "decimalPlaces": 0,
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~world-bank-indicators-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "countries": ["USA", "GBR"],
 "indicators": ["NY.GDP.MKTP.CD"],
 "fromYear": 2015
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/world-bank-indicators-scraper').call({
 countries: ['USA', 'GBR', 'DEU'],
 indicators: ['NY.GDP.MKTP.CD', 'SP.POP.TOTL'],
 fromYear: 2020,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const r of items) console.log(`${r.countryName} ${r.year} ${r.indicatorCode}: ${r.value}`);
```

### Notes

- **The response is `[metadata, rows]`, not an object.** Every World Bank response is a two-element array. This Actor handles it explicitly. It is the single most common way naive parsers break on this API.
- **Bad codes come back as HTTP 200.** An unknown country or indicator returns `[{ "message": [.] }]` with a success status, so the body is inspected rather than the status code. Those indicators land in `RUN_SUMMARY.failures` with the API's own message.
- **`region` and `incomeLevel` are joined, not returned.** The indicator endpoint does not carry them; the `/country` reference table is fetched once per run and joined on ISO code. If that lookup fails the run continues with both fields `null`.
- **`unit` is almost always empty.** The World Bank ships the unit inside `indicatorName` (`GDP (current US$)`) rather than in the `unit` field. It is preserved for the rare series that populate it.
- **Null values are dropped by default.** The API emits a row per country-year regardless of whether an observation exists; `skipNullValues: false` keeps them if you need a dense grid.
- **Aggregates are included when you ask for `all`.** Rows like `AFE` (Africa Eastern and Southern), `EMU` (euro area) or `WLD` (World) are aggregates, not countries, 78 of the 295 entries. They are marked with `region: "Aggregates"` and `incomeLevel: "Aggregates"`, so `region !== "Aggregates"` gives you sovereign states only.
- **Transient errors are retried.** 429, 5xx and network errors get four attempts with exponential backoff (0.5 s → 1 s → 2 s). A rejected code fails immediately rather than burning retries.
- **Public data only.** No authentication, no personal data, no access-control bypass.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need an account on the source?** No, and there is no credentials to manage.

**What happens if one indicator is unavailable?** It is reported in `RUN_SUMMARY.failures` and the run continues with the rest. The Actor only errors out if every indicator failed.

**Can I schedule it?** Yes, it is designed for scheduled runs. The World Bank republishes on a slow cadence, so weekly or monthly is usually enough.

**How many indicators can I pass at once?** There is no hard cap. Each indicator is one request per 20,000 rows, so dozens per run is normal.

**Why is the most recent year missing for some countries?** The World Bank publishes on a lag that varies by country and series. That is the source's cadence, not a scraping gap.

**How do I exclude regional aggregates?** Drop rows where `region` is `"Aggregates"`. The World Bank marks all 78 aggregate entities that way in both `region` and `incomeLevel`.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `countries` (type: `array`):

ISO-2 or ISO-3 country codes, e.g. 'US', 'GBR'. Aggregates work too. 'EMU' for the euro area, 'WLD' for the world. Leave empty (or pass 'all') for every country and aggregate.

## `indicators` (type: `array`):

World Bank indicator codes, e.g. 'NY.GDP.MKTP.CD' for GDP in current US$, 'SP.POP.TOTL' for population, 'FP.CPI.TOTL.ZG' for inflation. Full data.worldbank.org URLs are accepted and normalized.

## `fromYear` (type: `integer`):

Earliest year to include. Most series begin in 1960. Leave empty for the full available history.

## `toYear` (type: `integer`):

Latest year to include. Leave empty for the most recent year published.

## `skipNullValues` (type: `boolean`):

The World Bank returns a row for every country-year even when no observation exists. Leave on to drop those empty rows; turn off to keep a complete, gap-marked grid.

## `maxResults` (type: `integer`):

Cap the total number of records saved. Set 0 for no limit. One indicator across all countries and all years is roughly 17,000 rows.

## Actor input object example

```json
{
  "countries": [
    "US",
    "GB",
    "BD",
    "all"
  ],
  "indicators": [
    "NY.GDP.MKTP.CD",
    "https://data.worldbank.org/indicator/SP.POP.TOTL"
  ],
  "fromYear": 2000,
  "toYear": 2024,
  "skipNullValues": true,
  "maxResults": 0
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "countries": [
        "USA",
        "GBR",
        "DEU"
    ],
    "indicators": [
        "NY.GDP.MKTP.CD",
        "SP.POP.TOTL"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/world-bank-indicators-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "countries": [
        "USA",
        "GBR",
        "DEU",
    ],
    "indicators": [
        "NY.GDP.MKTP.CD",
        "SP.POP.TOTL",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/world-bank-indicators-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "countries": [
    "USA",
    "GBR",
    "DEU"
  ],
  "indicators": [
    "NY.GDP.MKTP.CD",
    "SP.POP.TOTL"
  ]
}' |
apify call arman-bd/world-bank-indicators-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/world-bank-indicators-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Uty7jhSBRhpnjBgmy/builds/eKmNGokzBnuqyv5V2/openapi.json