# World Bank Scraper · Indicators, Countries, Values & Series (`reapx/world-bank-scraper`) Actor Scrape World Bank global development indicators, country GDP, population, inflation, economic series, and historical annual values. Fast HTTP API scraper with pay-per-event pricing. - **URL**: https://apify.com/reapx/world-bank-scraper.md - **Developed by:** [Tarek Etman](https://apify.com/reapx) (community) - **Categories:** Business, Education, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.70 / 1,000 record returneds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![reapX — public sources in, addressable records out](https://reapx.dev/reapx.gif) ## World Bank Scraper · Indicators, Countries, Values & Series Scrape official World Bank global development indicators, country economic metrics, GDP figures, population stats, inflation rates, labor metrics, and multi-year time-series datasets directly from the World Bank Open Data REST API. > Maintained by **reapX**. Every row cites the Apify run that produced it — nothing is > inferred, modelled or filled in, and a field absent from the source is absent from the row. > The extracted archive for this source is browsable at > [reapx.dev/data/world-bank-scraper/](https://reapx.dev/data/world-bank-scraper/) and mirrored as an open dataset on > [Hugging Face](https://huggingface.co/datasets/reapxdev/world-bank-scraper) and > [Kaggle](https://www.kaggle.com/datasets/reapxdev/world-bank-scraper). Questions: reapxdev@proton.me *** ### Features & Capabilities - **Official Source**: Direct REST API integration with World Bank Open Data API (`api.worldbank.org/v2`). - **Comprehensive Coverage**: Access over 16,000 indicator series covering 260+ countries, territories, and regional/income aggregates. - **Flexible Filters**: Filter by indicator code, 2-character / 3-character ISO country code, regional cohorts (`EAP`, `ECA`, `LAC`, `SSA`, `EUU`, `OED`), income groups (`HIC`, `UMC`, `LMC`, `LIC`), or single years / date ranges (`2010:2024`). - **Entity Addressable**: Output rows include sanitized `companyName` (country or aggregate entity name) ensuring total 1:1 entity page generation and data linkage. - **Pay-Per-Event Pricing**: Pay purely per complete record returned ($0.001/record with tier discounts). Failed requests or empty queries are never charged. - **High Performance**: Lightweight HTTP scraper (no browser overhead) capable of streaming hundreds of indicator records per second. *** ### ⬇️ Input The actor accepts clean structured input to target specific indicators, country cohorts, and time horizons. | Parameter | Type | Default / Prefill | Description | | :--- | :--- | :--- | :--- | | `indicator` | String | `NY.GDP.MKTP.CD` | World Bank indicator code (e.g. `NY.GDP.MKTP.CD` for GDP, `SP.POP.TOTL` for Population, `FP.CPI.TOTL.ZG` for Inflation). | | `country` | String | `all` | Country ISO code (`USA`, `DEU`, `CHN`), regional aggregate (`EAP`, `EUU`, `SSA`), income level (`HIC`, `LIC`), or `all`. | | `date` | String | `2015:2024` | Single year (`2023`) or range (`2010:2024`) controlling temporal scope. | | `source` | Integer | `2` | World Bank catalog source ID (`2` for World Development Indicators WDI). | | `max_results` | Integer | `500` | Hard cap on total indicator observation records to return. | #### Input Example (JSON) ```json { "indicator": "NY.GDP.PCAP.CD", "country": "all", "date": "2015:2024", "source": 2, "max_results": 1000 } ``` *** ### ⬆️ Output All extracted records are pushed directly to the run's default dataset. Each row corresponds to a single annual observation for a country/entity. | Field Name | Type | Example | Description | | :--- | :--- | :--- | :--- | | `companyName` | String | `United States` | Primary addressable entity key (country or aggregate region name). | | `countryCode` | String | `US` | 2-letter or aggregate region code. | | `countryIso3Code` | String | `USA` | 3-letter ISO 3166-1 alpha-3 code. | | `indicatorId` | String | `NY.GDP.PCAP.CD` | World Bank indicator series code. | | `indicatorName` | String | `GDP per capita (current US$)` | Human-readable series title. | | `year` | String | `2023` | Four-digit observation year. | | `value` | Number | `81632.25` | Recorded numerical value (or null if missing in source). | | `unit` | String | `""` | Unit of measurement if specified by World Bank. | | `obsStatus` | String | `""` | Source observation status tag. | | `decimal` | Integer | `0` | Decimal precision reported by World Bank. | #### Output Row Example (JSON) ```json { "companyName": "United States", "countryCode": "US", "countryIso3Code": "USA", "indicatorId": "NY.GDP.PCAP.CD", "indicatorName": "GDP per capita (current US$)", "year": "2023", "value": 81632.251953125, "unit": "", "obsStatus": "", "decimal": 0 } ``` *** ### How it works 1. **API Initialization**: The scraper queries `https://api.worldbank.org/v2/country/{country}/indicator/{indicator}` using HTTP requests. 2. **Metadata Count**: Performs an initial lightweight metadata query to verify total matching records and validate parameters before scraping. 3. **Paginated Retrieval**: Streams items page-by-page (500 items per request) with automatic exponential backoff on network errors or 429/5xx status codes. 4. **Data Normalization**: Cleans and sanitizes country labels into `companyName` values matching exact entity page regex specifications. 5. **Pay-Per-Event Emission**: Pushes records to the default dataset while executing atomic pay-per-event micro-charging ($0.001 per record). *** ### ❓ FAQ #### What indicators can I scrape? You can scrape any valid World Bank indicator code from the World Development Indicators (WDI) catalog or other databases. Popular examples include: - `NY.GDP.MKTP.CD`: GDP (current US$) - `NY.GDP.PCAP.CD`: GDP per capita (current US$) - `NY.GDP.MKTP.KD.ZG`: GDP growth (annual %) - `SP.POP.TOTL`: Total Population - `FP.CPI.TOTL.ZG`: Inflation rate (annual %) - `SL.UEM.TOTL.ZS`: Unemployment rate (% of labor force) - `EN.ATM.CO2E.PC`: CO2 emissions (metric tons per capita) - `SP.DYN.LE00.IN`: Life expectancy at birth (years) #### How is pricing calculated? This actor operates on Pay-Per-Event pricing. You are charged $0.001 per complete record pushed to the dataset. Platform compute usage is absorbed. Failed requests, rate limits, or queries matching 0 rows incur zero charge. #### How do I filter by specific country or region? Set the `country` parameter to a 3-letter ISO code (e.g. `USA`, `DEU`, `JPN`, `BRA`) or a regional cohort code (e.g. `EAP` for East Asia & Pacific, `EUU` for European Union, `HIC` for High Income countries). Leave as `all` to pull all available nations and aggregates. #### Can I run this scraper programmatically? Yes! Use the Apify Python SDK, JavaScript SDK, or standard HTTP client to invoke this actor via the Apify API. ```python from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("reapx/world-bank-scraper").call(run_input={ "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2020:2024", "max_results": 500 }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["companyName"], item["year"], item["value"]) ``` *** ### 💬 Your feedback Have feature suggestions, bug reports, or requests for custom dataset extractions? Contact the maintainers directly at **reapxdev@proton.me** or visit [reapx.dev](https://reapx.dev). *** *Disclaimer: Unofficial - not affiliated with World Bank. Collects public data only. reapx. Contact reapxdev@proton.me.* ### 🧪 Example input A real, runnable configuration — this is an actual input this Actor has run with. ```json { "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2020:2024", "source": 2, "max_results": 500 } ``` ### 📄 Sample output One real row from a real run of this Actor, unedited. ```json { "companyName": "Africa Eastern and Southern", "countryCode": "ZH", "countryIso3Code": "AFE", "indicatorId": "NY.GDP.MKTP.CD", "indicatorName": "GDP (current US$)", "year": "2022", "value": 1226461319482.05, "unit": "", "obsStatus": "", "decimal": 0 } ``` ### ⚠️ Run outcomes and error handling This Actor reports what happened in the run's **status message**, and it always keeps whatever it collected. These are the outcomes you can get and what each one means. | Outcome | What it means | |---|---| | **Success** | Rows were returned and you were charged `record-returned` at $0.001 per row. | | **No matches** | The source returned nothing for your filters. **Nothing is charged.** Widen the date window or drop a filter. | | **Partial - source refused** | The source rate-limited or refused some requests. The affected items are skipped and named in the log, and **everything already collected is still pushed**. A block never discards a run's work. | | **Rejected filter** | The source itself rejected the filter combination. The run fails fast with the source's own reason and **nothing is charged.** | #### What is guaranteed either way - **Every row is pushed as it is built**, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not. - **A field absent from the source is absent from the row.** Nothing is inferred, modelled or filled in to make a row look complete. # Actor input Schema ## `indicator` (type: `string`): World Bank indicator code (e.g. NY.GDP.MKTP.CD for GDP in current US$, SP.POP.TOTL for Population, FP.CPI.TOTL.ZG for Inflation %).

Consequence: Determines which economic, social, or environmental metric is extracted across countries. Empty value: Defaults to NY.GDP.MKTP.CD.

Leave this field empty to skip this filter. ## `country` (type: `string`): Filter by 2-character/3-character ISO country code (e.g. USA, DEU, CHN), regional aggregate (e.g. EAP, EUU, SSA), income level (e.g. HIC, LIC), or all.

Consequence: Narrows economic series extraction to a specific nation or regional cohort. Empty value: Defaults to all.

Leave this field empty to skip this filter. ## `date` (type: `string`): Filter historical data by single year (e.g. 2023) or year range formatted as YYYY:YYYY (e.g. 2010:2024 or 2000:2023).

Consequence: Controls temporal coverage of returned indicator values; broader ranges return more rows. Empty value: Defaults to 2015:2024.

Leave this field empty to skip this filter. ## `source` (type: `integer`): Numeric ID of World Bank database catalog (e.g. 2 for World Development Indicators WDI, 11 for Africa Development Indicators).

Consequence: Selects underlying database source catalog for indicator definitions. Empty value: Defaults to 2 (WDI). ## `max_results` (type: `integer`): Maximum number of indicator observation records to extract and save to the default dataset (e.g. 500 or 2000).

Consequence: Higher limits harvest larger cross-country series but extend execution time and pay-per-event cost. Empty value: Defaults to 500.

The higher the number, the longer the run takes and the more you are charged, because you pay per row returned. ## Actor input object example ```json { "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2015:2024", "source": 2, "max_results": 500 } ``` # Actor output Schema ## `results` (type: `string`): Every World Bank indicator observation record found by this run, one row per item, in the default dataset. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2015:2024" }; // Run the Actor and wait for it to finish const run = await client.actor("reapx/world-bank-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2015:2024", } # Run the Actor and wait for it to finish run = client.actor("reapx/world-bank-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "indicator": "NY.GDP.MKTP.CD", "country": "all", "date": "2015:2024" }' | apify call reapx/world-bank-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,reapx/world-bank-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/FMgEchTdSnybxvjQ3/builds/j6hk3AsjDYtJNqGcG/openapi.json