World Bank Indicators Scraper: WDI by Country & Year

Collect yearly development indicators—such as total population, GDP, and economic metrics—for targeted countries across specified date ranges. This tool gathers structured historical records in bulk, allowing you to export global datasets directly to JSON, CSV, or Excel formats.

Rather than searching through individual web pages, you can extract reliable, multi-year economic and demographic metrics in clean, structured tables.

Use Cases

• Population trends: Extract multi-year population totals (SP.POP.TOTL) for specific countries or across all supported nations.
• Economic analysis: Pull GDP figures (NY.GDP.MKTP.CD) and compare national economic metrics year over year.
• Custom date windows: Filter dataset extractions to specific ranges (e.g., 2000:2024) to eliminate unnecessary historical data.
• Automated reporting: Maintain up-to-date research spreadsheets and databases with scheduled data refreshes.
• Research & analysis: Keep indicator labels, country identifiers, observation statuses, and numeric metrics side-by-side.

Extracted Data Points

Each observation includes standard metadata for clean analysis:

• Country name, standard country code, and ISO3 identifier
• Indicator code and indicator name
• Year or observation period
• Metric value (including decimal precision and observation status)

Input Parameters

Parameter	Type	Default	Description
mode	String	values	Extraction mode (one row per country, indicator, and year).
countries	Array	["US"]	ISO 2-letter or 3-letter country codes (or ["all"] for all available countries).
indicators	Array	["SP.POP.TOTL"]	Target indicator codes (one per entry).
date	String	empty	Target year or range (e.g., 2000:2024). Leave blank to retrieve all available years.
maxItems	Integer	100	Maximum row count limit (up to 1,000).
timeoutSecs	Integer	300	Overall execution limit in seconds.

Example Input

{
    "mode": "values",
    "countries": ["US"],
    "indicators": ["SP.POP.TOTL"],
    "date": "2000:2024",
    "maxItems": 100,
    "timeoutSecs": 300
}

Output Format

{
    "indicator": {
        "id": "SP.POP.TOTL",
        "value": "Population, total"
    },
    "indicatorId": "SP.POP.TOTL",
    "indicatorName": "Population, total",
    "country": {
        "id": "US",
        "value": "United States"
    },
    "countryId": "US",
    "countryName": "United States",
    "countryiso3code": "USA",
    "date": "2025",
    "value": 341784857,
    "unit": "",
    "obs_status": "",
    "decimal": 0
}

How It Works

1. Processes input configurations, applying country lists, indicator codes, and year range filters.
2. Formats requests into structured pages to collect observations efficiently.
3. Automatically deduplicates rows by indicator, country, and year.
4. Handles brief network interruptions smoothly using retry delays.
5. Generates clean datasets available for instant export as JSON, CSV, or Excel.

Frequently Asked Questions

Can I pull data for all countries at once?

Yes. Include "all" in the countries array. Because this generates a large volume of data, specifying a date range and maxItems limit is recommended.

What happens if a specific indicator code is invalid?

An error entry will be recorded for that query explaining the issue, allowing the process to continue running for the remaining valid indicators.

Why are some values empty or null?

If the source source dataset does not have a recorded metric for a specific year, the row is preserved with a null value so data gaps remain visible in your analysis.

---

Legal & Compliance

Trademark Notice

World Bank Indicators and related names, logos, and trademarks belong to their respective owners. Their mention in this documentation is strictly to identify data compatibility and utility, and does not imply affiliation, sponsorship, or endorsement.

Disclaimer

This tool is an independent utility and is not affiliated with, endorsed by, or officially connected to third-party data providers. Users are responsible for ensuring their usage complies with relevant terms of service, usage guidelines, and licensing terms.

Is web scraping legal?

This tool collects publicly available statistical data without bypassing access controls. Collecting public data is standard practice, but proper compliance requires considering how the data is used:

• Personal data: Ensure dataset usage respects regional privacy regulations (such as GDPR or CCPA) when combining outputs with personal user records.
• Copyright & Terms: Respect the terms of use, usage limits, and licensing terms set by the data source.
• Legal advice: This summary is provided for general informational purposes and does not constitute formal legal counsel.