Pricing

$2.00 / 1,000 results

Try for free

Go to Apify Store

Japan world bank indicators Prices — Listings & Market Data

Try for free

Scrape World Bank development indicators and statistical data. Returns structured macroeconomic and social indicator data for countries worldwide.

Pricing

$2.00 / 1,000 results

Rating

0.0

(0)

Developer

atushi ino

Actor stats

Bookmarked

Total users

Monthly active users

15 days ago

Last modified

Routing Number Lookup

automly/bank-routings-api

Look up and validate US bank routing (ABA) numbers: bank name, address, phone, transfer types and status for any routing number, or find routing numbers by bank name and ZIP code. Export to CSV, JSON or Excel, schedule runs or use the API.

Automly

🏦 Bank Routing Number Lookup

easyapi/bank-routing-number-lookup

Effortlessly retrieve and validate US bank information using routing numbers. Get comprehensive details including bank names, addresses, and institution information for ACH/WIRE transfers. Perfect for payment processing systems and financial data verification.

EasyApi

World Bank WDI Economic Indicators Data API

koalastuff/world-bank-wdi-explorer

Retrieve World Bank WDI indicators for countries and years. Export GDP, population, health, trade, and development data for analysis and dashboards.

Timo Schmidt

World Bank Data Scraper - Economic Indicators API

pink_comic/world-bank-indicators-search

Scrape World Bank Open Data for GDP, population, inflation, poverty, CO2, trade, health, education, and 29,000+ development indicators across 200+ countries. Export annual time-series data for economic research, ESG reporting, policy analysis, market intelligence, and AI/RAG workflows.

Ava Torres

Japan upgarage parts Prices — Listings & Market Data

fruitful_quintessence/upgarage-parts-scraper

Scrape UpGarage auto parts and accessories listings. Get structured pricing data for used and new car parts from Japan's automotive aftermarket retailer.

atushi ino

Japan golfpartner used club Prices — Listings & Market Data

fruitful_quintessence/golfpartner-used-club-scraper

Scrape Golf Partner used golf club and equipment listings. Returns structured pricing and specification data for clubs, shoes, and accessories.

atushi ino

World Bank Development Indicators

wakey7dev/world-bank-indicators

Extract World Bank development data — GDP, population, poverty, education, health for all countries. Economic research, policy analysis, market intelligence.

Chris Wakefield

World Bank World Development Indicators

prophed/world-bank-world-development-indicators

Fetch and normalize the World Bank indicator catalog for World Development Indicators.

Prophed Com

Japan goobike motorcycle Prices — Listings & Market Data

fruitful_quintessence/goobike-motorcycle-scraper

Scrape GooBike motorcycle and scooter listings. Returns structured pricing and specification data from Japan's two-wheeled vehicle marketplace.

atushi ino

World Bank Indicators Scraper — No Login, $1.86/1k

velvety_bedbug/world-bank-indicators-scraper

Fetches economic indicator data from the World Bank API. Access GDP, population, trade, health, and hundreds of other development indicators for any country. Free, no API key required. No login or cookies. MCP-ready for AI agents. $1.86 per 1,000 records.

Peters Bugs

{ "title": "World Bank Indicators input", "type": "object", "schemaVersion": 1, "properties": { "countries": { "title": "Country filter (comma-separated ISO3 codes)", "type": "string", "description": "Filter by country codes (e.g. 'USA,JPN,DEU'). Leave empty for all 50.", "editor": "textfield" }, "indicators": { "title": "Indicator filter (comma-separated codes)", "type": "string", "description": "Filter by indicator codes (e.g. 'NY.GDP.MKTP.CD,SP.POP.TOTL'). Leave empty for all 7.", "editor": "textfield" }, "limit": { "title": "Max results", "type": "integer", "description": "Maximum records to return (1-100000).", "default": 10000, "minimum": 1, "maximum": 100000, "editor": "number" } } }

{ "actorSpecification": 1, "name": "world-bank-indicators", "title": "World Bank Indicators (50 countries × 7 indicators, 2015-2024)", "description": "World Bank official API v2 for 50 countries × 7 key indicators: GDP, GDP per capita, population, inflation, unemployment, life expectancy, external balance. 2015-2024 time series. No API key. Ideal for economists, researchers, and AI agents doing country analysis.", "version": "0.1", "buildTag": "latest", "dockerfile": "./Dockerfile", "input": "./INPUT_SCHEMA.json", "outputSchema": { "actorOutputSchemaVersion": 1, "title": "World Bank indicator record", "type": "object", "properties": { "source": { "type": "string", "description": "Source ID (worldbank)", "template": "{{source}}" }, "countryCode": { "type": "string", "description": "ISO-3 country code (e.g. USA, JPN)", "template": "{{countryCode}}" }, "countryName": { "type": "string", "description": "Country name", "template": "{{countryName}}" }, "indicatorCode": { "type": "string", "description": "Indicator code (e.g. NY.GDP.MKTP.CD)", "template": "{{indicatorCode}}" }, "indicatorName": { "type": "string", "description": "Indicator name", "template": "{{indicatorName}}" }, "year": { "type": "string", "description": "Year (YYYY)", "template": "{{year}}" }, "value": { "type": "string", "description": "Numeric value", "template": "{{value}}" }, "unit": { "type": "string", "description": "Unit (USD, persons, %, years)", "template": "{{unit}}" }, "rawData": { "type": "string", "description": "Original JSON row from World Bank (string)", "template": "{{rawData}}" } } }, "minMemoryMbytes": 256 }

1"""World Bank Indicators Apify Actor. 2 3世界銀行 (World Bank) v2 API から主要指標 (GDP / 人口 / インフレ / 失業率 / 寿命 / 経常収支) を 4主要50ヶ国について 2015-2024 で取得し、Dataset に出力する. 5""" 6from __future__ import annotations 7 8import asyncio 9import json 10from typing import Any 11 12import httpx 13from apify import Actor 14 15from src.sources.worldbank import ( 16 WORLDBANK_COUNTRIES, 17 WORLDBANK_INDICATORS, 18 WorldBankSource, 19) 20 21 22def _to_row(rec: Any) -> dict[str, str]: 23 out: dict[str, str] = {} 24 for k, v in rec.model_dump().items() if hasattr(rec, "model_dump") else rec.dict().items(): 25 if v is None: 26 out[k] = "" 27 elif isinstance(v, (list, dict)): 28 out[k] = json.dumps(v, ensure_ascii=False) 29 else: 30 out[k] = str(v) 31 return out 32 33 34async def main() -> None: 35 async with Actor: 36 run_input = await Actor.get_input() or {} 37 limit = int(run_input.get("limit") or 10000) 38 countries_filter = (run_input.get("countries") or "").strip() 39 indicators_filter = (run_input.get("indicators") or "").strip() 40 41 countries = WORLDBANK_COUNTRIES 42 if countries_filter: 43 wanted = {c.strip().upper() for c in countries_filter.split(",")} 44 countries = [c for c in countries if c["code"] in wanted] 45 indicators = WORLDBANK_INDICATORS 46 if indicators_filter: 47 wanted_ind = {i.strip() for i in indicators_filter.split(",")} 48 indicators = [i for i in indicators if i["code"] in wanted_ind] 49 50 src = WorldBankSource(countries=countries, indicators=indicators) 51 Actor.log.info(f"Fetching {len(countries)} countries × {len(indicators)} indicators") 52 53 rows: list[dict[str, str]] = [] 54 # 同期 client を async から呼ぶ (httpx.Client) 55 with httpx.Client(timeout=30) as client: 56 for rec in src.fetch(client): 57 rows.append(_to_row(rec)) 58 if len(rows) >= limit: 59 break 60 61 Actor.log.info(f"Got {len(rows)} records") 62 63 # クエリ情報も先頭に 64 meta = {"__meta__": "1", "input": json.dumps(run_input, ensure_ascii=False)} 65 rows.insert(0, meta) 66 67 dataset = await Actor.open_dataset() 68 await dataset.push_data(rows) 69 Actor.log.info(f"Pushed {len(rows)} rows to dataset") 70 71 72if __name__ == "__main__": 73 asyncio.run(main())

1"""Pydantic モデル定義. 2 3API レスポンス・内部データ構造を統一するために使う. 4mypy strict 通過を念頭に、フィールド型とデフォルト値を明示する. 5""" 6from __future__ import annotations 7 8from datetime import datetime 9from enum import Enum 10from typing import Any, Optional 11 12from pydantic import BaseModel, Field 13 14 15class DataSource(str, Enum): 16 """データソース識別子.""" 17 18 WORLDBANK = "worldbank" 19 OECD = "oecd" 20 IMF = "imf" 21 UN_DATA = "un_data" 22 EUROSTAT = "eurostat" 23 OPEN_METEO = "open_meteo" 24 25 26# ===== 内部レコード (collector → DB) ===== 27 28 29class RawIndicatorRecord(BaseModel): 30 """collector が取得して DB に保存する前段階のレコード. 31 32 Attributes: 33 source: データソース識別子. 34 country_code: ISO-3 国コード (例: 'JPN', 'USA'). 35 country_name: 表示用国名 (例: 'Japan'). 36 indicator_code: 指標コード (例: 'NY.GDP.MKTP.CD'). 37 indicator_name: 指標名 (例: 'GDP (current US$)'). 38 year: 年度 (4 桁 int). 時系列データ用. 39 value: 指標値 (float or None). 40 unit: 単位 (例: 'USD', 'persons', '%'). 41 raw_data: 元 JSON (デバッグ用). 42 """ 43 44 source: DataSource = Field(..., description="データソース識別子.") 45 country_code: str = Field(..., description="ISO-3 国コード.") 46 country_name: str = Field(..., description="表示用国名.") 47 indicator_code: str = Field(..., description="指標コード.") 48 indicator_name: str = Field(..., description="指標名.") 49 year: int = Field(..., description="年度 (4 桁).") 50 value: Optional[float] = Field(None, description="指標値.") 51 unit: Optional[str] = Field(None, description="単位.") 52 raw_data: dict[str, Any] = Field(default_factory=dict) 53 54 55class Country(BaseModel): 56 """国情報.""" 57 58 code: str = Field(..., description="ISO-3 国コード (例: 'JPN').") 59 name: str = Field(..., description="国名 (例: 'Japan').") 60 region: Optional[str] = Field(None, description="地域 (例: 'Asia').") 61 sources: list[str] = Field( 62 default_factory=list, description="この国をカバーするソース一覧." 63 ) 64 65 66class Indicator(BaseModel): 67 """指標メタ情報.""" 68 69 code: str = Field(..., description="指標コード.") 70 name: str = Field(..., description="指標名.") 71 source: str = Field(..., description="提供元ソース.") 72 unit: Optional[str] = Field(None, description="単位.") 73 description: Optional[str] = Field(None, description="指標説明.") 74 75 76class IndicatorPoint(BaseModel): 77 """1 国 × 1 指標 × 1 年の 1 データ点.""" 78 79 source: str 80 country_code: str 81 country_name: str 82 indicator_code: str 83 indicator_name: str 84 year: int 85 value: Optional[float] 86 unit: Optional[str] 87 88 89class CountryIndicatorsResponse(BaseModel): 90 """GET /countries/{CC}/indicators のレスポンス.""" 91 92 country_code: str 93 country_name: str 94 indicator: str 95 points: list[IndicatorPoint] 96 97 98class CompareResponse(BaseModel): 99 """GET /compare のレスポンス.""" 100 101 indicator: str 102 countries: list[str] 103 items: list[IndicatorPoint] 104 105 106class CountryListResponse(BaseModel): 107 """GET /countries のレスポンス.""" 108 109 total: int 110 items: list[Country] 111 112 113class IndicatorListResponse(BaseModel): 114 """GET /indicators のレスポンス.""" 115 116 total: int 117 items: list[Indicator] 118 119 120# ===== 気象 (Open-Meteo) ===== 121 122 123class WeatherCurrent(BaseModel): 124 """現在の気象.""" 125 126 temperature_2m: Optional[float] = Field(None, description="気温 (℃).") 127 relative_humidity_2m: Optional[float] = Field(None, description="相対湿度 (%).") 128 apparent_temperature: Optional[float] = Field(None, description="体感温度 (℃).") 129 precipitation: Optional[float] = Field(None, description="降水量 (mm).") 130 wind_speed_10m: Optional[float] = Field(None, description="風速 (m/s).") 131 weather_code: Optional[int] = Field(None, description="WMO 天気コード.") 132 133 134class WeatherDaily(BaseModel): 135 """日次気象.""" 136 137 time: list[str] = Field(default_factory=list, description="日付リスト (YYYY-MM-DD).") 138 temperature_2m_max: list[Optional[float]] = Field(default_factory=list) 139 temperature_2m_min: list[Optional[float]] = Field(default_factory=list) 140 precipitation_sum: list[Optional[float]] = Field(default_factory=list) 141 142 143class WeatherResponse(BaseModel): 144 """GET /weather のレスポンス.""" 145 146 latitude: float 147 longitude: float 148 timezone: Optional[str] = None 149 current: WeatherCurrent 150 daily: WeatherDaily 151 152 153# ===== ソース情報 ===== 154 155 156class SourceInfo(BaseModel): 157 """データソース情報.""" 158 159 id: str 160 name: str 161 url: str 162 coverage: str 163 license: str 164 description: str 165 166 167class SourcesResponse(BaseModel): 168 """GET /sources のレスポンス.""" 169 170 items: list[SourceInfo] 171 172 173# ===== ヘルスチェック ===== 174 175 176class HealthResponse(BaseModel): 177 """GET / のレスポンス.""" 178 179 status: str 180 version: str 181 country_count: int 182 indicator_count: int 183 record_count: int 184 last_update_at: Optional[str] = None 185 186 187def utcnow_iso() -> str: 188 """UTC の現在時刻を ISO 8601 文字列で返す.""" 189 return datetime.utcnow().replace(microsecond=0).isoformat() + "Z"

1"""ノーマライザー. 2 3異なるソース (World Bank / OECD / IMF / UN / Eurostat) の国コードや 4指標コードを統一形式に正規化するヘルパー群. 5""" 6from __future__ import annotations 7 8import re 9from typing import Optional 10 11 12# ISO-3 国コード検証用 (3 文字の大文字アルファベット) 13_ISO3_PATTERN = re.compile(r"^[A-Z]{3}$") 14 15 16# World Bank / OECD 共通の代表的指標コードマッピング 17# (UI で扱う短い名前 → 内部で使う正式コード) 18INDICATOR_ALIASES: dict[str, dict[str, str]] = { 19 "worldbank": { 20 "GDP": "NY.GDP.MKTP.CD", 21 "GDP_PER_CAPITA": "NY.GDP.PCAP.CD", 22 "POPULATION": "SP.POP.TOTL", 23 "INFLATION": "FP.CPI.TOTL.ZG", 24 "UNEMPLOYMENT": "SL.UEM.TOTL.ZS", 25 "LIFE_EXPECTANCY": "SP.DYN.LE00.IN", 26 "TRADE_BALANCE": "NE.RSB.GNFS.CD", 27 }, 28 "oecd": { 29 "GDP": "GDP.Q.Q", 30 "GDP_PER_CAPITA": "GDPVD.Q", 31 "UNEMPLOYMENT": "ALFS.Q", 32 "INFLATION": "CPI.Q", 33 }, 34 "imf": { 35 "GDP": "NGDP_RPCH", 36 "GDP_PER_CAPITA": "NGDPDPC", 37 "INFLATION": "PCPI_PCH", 38 "UNEMPLOYMENT": "LUR_PT", 39 }, 40 "eurostat": { 41 "GDP": "namq_10_gdp", 42 "POPULATION": "demo_pjan", 43 "INFLATION": "tec00118", 44 "UNEMPLOYMENT": "une_rt_a", 45 }, 46} 47 48 49def normalize_country_code(raw: str) -> Optional[str]: 50 """国コードを ISO-3 大文字に正規化. 51 52 Args: 53 raw: 元の国コード文字列. 54 55 Returns: 56 正規化されたコード. 無効なら None. 57 58 Examples: 59 >>> normalize_country_code("jpn") 60 'JPN' 61 >>> normalize_country_code(" USA ") 62 'USA' 63 >>> normalize_country_code("XX") 64 None 65 """ 66 if not raw: 67 return None 68 code = raw.strip().upper() 69 if _ISO3_PATTERN.match(code) is None: 70 return None 71 return code 72 73 74def normalize_indicator_code( 75 source: str, indicator: str, 76) -> tuple[str, bool]: 77 """指標コードを内部形式に正規化. 78 79 短い名前 (例: 'GDP') の場合は正式コードに解決する. 80 81 Args: 82 source: ソース名. 83 indicator: 指標コードまたはエイリアス. 84 85 Returns: 86 (正規化済みコード, エイリアス経由で解決されたかどうか). 87 88 Examples: 89 >>> normalize_indicator_code("worldbank", "GDP") 90 ('NY.GDP.MKTP.CD', True) 91 >>> normalize_indicator_code("worldbank", "NY.GDP.MKTP.CD") 92 ('NY.GDP.MKTP.CD', False) 93 """ 94 if not indicator: 95 return "", False 96 aliases = INDICATOR_ALIASES.get(source, {}) 97 if indicator in aliases: 98 return aliases[indicator], True 99 return indicator, False 100 101 102def coerce_int(value: object) -> Optional[int]: 103 """値を int に変換. 失敗時 None.""" 104 if value is None: 105 return None 106 try: 107 result = int(str(value)) 108 except (TypeError, ValueError): 109 return None 110 return result 111 112 113def safe_float(value: object) -> Optional[float]: 114 """値を float に変換. 失敗時 None.""" 115 if value is None: 116 return None 117 if isinstance(value, str) and value.strip() in ("", "NA", "N/A", "null", "None"): 118 return None 119 try: 120 result = float(str(value)) 121 except (TypeError, ValueError): 122 return None 123 return result if isinstance(result, float) else None 124 125 126def safe_str(value: object, default: str = "") -> str: 127 """値を str に変換. 失敗時 default.""" 128 if value is None: 129 return default 130 if not isinstance(value, str): 131 try: 132 return str(value) 133 except Exception: 134 return default 135 return value 136 137 138def region_for_country(code: str) -> str: 139 """ISO-3 国コードからおおまかな地域を返す. 140 141 簡易分類 (完全ではない). EU/主要国は地域を返す. 142 該当しない国は 'Other' を返す. 143 """ 144 # 主要 OECD/World Bank 分類 145 eu_codes = { 146 "AUT", "BEL", "BGR", "HRV", "CYP", "CZE", "DNK", "EST", "FIN", "FRA", 147 "DEU", "GRC", "HUN", "IRL", "ITA", "LVA", "LTU", "LUX", "MLT", "NLD", 148 "POL", "PRT", "ROU", "SVK", "SVN", "ESP", "SWE", 149 } 150 asia_codes = { 151 "JPN", "CHN", "KOR", "IND", "IDN", "THA", "VNM", "MYS", "SGP", 152 "PHL", "PAK", "BGD", "LKA", "NPL", "MMR", "KHM", "LAO", "MNG", 153 "AFG", "BTN", "BRN", "TLS", 154 } 155 middle_east = { 156 "SAU", "ARE", "ISR", "TUR", "IRN", "IRQ", "JOR", "LBN", "SYR", 157 "YEM", "OMN", "QAT", "KWT", "BHR", "PSE", 158 } 159 africa = { 160 "NGA", "ETH", "EGY", "ZAF", "KEN", "UGA", "TZA", "GHA", "DZA", 161 "MAR", "TUN", "SDN", "AGO", "MOZ", "ZMB", "ZWE", "CMR", "SEN", 162 "CIV", "MLI", "BFA", "MDG", "MWI", "RWA", "BEN", "TGO", "SLE", 163 "LBR", "NER", "COD", "COG", "GAB", "GNB", "GNQ", "MRT", "SWZ", 164 "LSO", "BDI", "ERI", "DJI", "SOM", "CAF", "TCD", "LBY", 165 } 166 americas = { 167 "USA", "CAN", "MEX", "BRA", "ARG", "CHL", "COL", "PER", "VEN", 168 "ECU", "BOL", "URY", "PRY", "CUB", "DOM", "GTM", "HND", "NIC", 169 "CRI", "PAN", "JAM", "HTI", "BHS", "TTO", "GUY", "SUR", "BLZ", 170 "SLV", 171 } 172 oceania = {"AUS", "NZL", "PNG", "FJI", "WSM", "TON", "VUT", "SLB"} 173 174 if code in eu_codes: 175 return "Europe (EU)" 176 if code in asia_codes: 177 return "Asia" 178 if code in middle_east: 179 return "Middle East" 180 if code in africa: 181 return "Africa" 182 if code in americas: 183 return "Americas" 184 if code in oceania: 185 return "Oceania" 186 return "Other" 187 188 189__all__ = [ 190 "INDICATOR_ALIASES", 191 "normalize_country_code", 192 "normalize_indicator_code", 193 "coerce_int", 194 "safe_float", 195 "safe_str", 196 "region_for_country", 197]

1"""データソース基底クラス (Actor用、DB 依存なし). 2 3fetch(client) のみを実装する. 4""" 5from __future__ import annotations 6 7import logging 8from dataclasses import dataclass, field 9from typing import Any, Iterable, Optional 10 11import httpx 12 13from ..models import DataSource, RawIndicatorRecord 14 15logger = logging.getLogger(__name__) 16 17 18@dataclass 19class FetchResult: 20 source: DataSource 21 record_count: int = 0 22 countries: list[dict[str, Any]] = field(default_factory=list) 23 indicators: list[dict[str, Any]] = field(default_factory=list) 24 error: Optional[str] = None 25 26 @property 27 def ok(self) -> bool: 28 return self.error is None 29 30 31class BaseSource: 32 source_id: DataSource = DataSource.WORLDBANK 33 default_url: str = "" 34 http_timeout: float = 30.0 35 36 def __init__(self, base_url: Optional[str] = None, timeout: float = 30.0) -> None: 37 self.base_url = base_url or self.default_url 38 self.http_timeout = timeout 39 40 def list_countries(self) -> list[dict[str, Any]]: 41 return [] 42 43 def list_indicators(self) -> list[dict[str, Any]]: 44 return [] 45 46 def fetch(self, client: httpx.Client) -> Iterable[RawIndicatorRecord]: 47 raise NotImplementedError 48 49 def get_client(self) -> httpx.Client: 50 return httpx.Client(timeout=self.http_timeout)

1"""Eurostat データソース. 2 3公式 v1 JSON-stat API: https://ec.europa.eu/eurostat/api/dissemination/statistics/1.0/data/ 4geo は ISO-3166 alpha-2 (DE/FR/JP...) 形式が正解。 5SDMX 2.1 API はドット区切りキーが必要で複雑なため v1 を使用する。 6""" 7from __future__ import annotations 8 9import logging 10from typing import Any, Iterable, Optional 11 12import httpx 13 14from ..models import DataSource, RawIndicatorRecord 15from ..normalizer import safe_float 16from .base import BaseSource 17 18logger = logging.getLogger(__name__) 19 20 21# 実際の EU 27 カ国 (ISO-3166 alpha-2。JSON-stat v1 API は ISO2 が正解) 22EUROSTAT_COUNTRIES: list[dict[str, str]] = [ 23 {"code": cc, "name": nm, "region": "Europe (EU)"} 24 for cc, nm in [ 25 ("AT", "Austria"), ("BE", "Belgium"), ("BG", "Bulgaria"), 26 ("CY", "Cyprus"), ("CZ", "Czechia"), ("DE", "Germany"), 27 ("DK", "Denmark"), ("EE", "Estonia"), ("ES", "Spain"), 28 ("FI", "Finland"), ("FR", "France"), ("GR", "Greece"), 29 ("HR", "Croatia"), ("HU", "Hungary"), ("IE", "Ireland"), 30 ("IT", "Italy"), ("LT", "Lithuania"), ("LU", "Luxembourg"), 31 ("LV", "Latvia"), ("MT", "Malta"), ("NL", "Netherlands"), 32 ("PL", "Poland"), ("PT", "Portugal"), ("RO", "Romania"), 33 ("SE", "Sweden"), ("SI", "Slovenia"), ("SK", "Slovakia"), 34 ] 35] 36 37# 27 国確認 (EU 加盟国) 38assert len(EUROSTAT_COUNTRIES) == 27, len(EUROSTAT_COUNTRIES) 39 40 41# データセット定義: dataset code, フィルタ, 指標メタ 42# 2026-09-05 実測: v1 JSON-stat API で 200 確認済みのフィルタのみ掲載 43EUROSTAT_DATASETS: list[dict[str, Any]] = [ 44 { 45 "code": "namq_10_gdp", 46 "name": "GDP and main components (quarterly)", 47 "indicator_code": "GDP", 48 "indicator_name": "GDP (chain linked volumes, quarterly)", 49 "unit": "CLV10_MEUR", 50 # namq_10_gdp の dims: [freq, unit, s_adj, na_item, geo, time] 51 "params": { 52 "unit": "CLV10_MEUR", 53 "s_adj": "SCA", 54 "na_item": "B1GQ", 55 }, 56 }, 57 { 58 "code": "demo_pjan", 59 "name": "Population on 1 January by age and sex", 60 "indicator_code": "POPULATION", 61 "indicator_name": "Population (total, 1 January)", 62 "unit": "persons", 63 # demo_pjan の dims: [freq, unit, age, sex, geo, time] 64 "params": {"sex": "T", "age": "TOTAL"}, 65 }, 66 { 67 "code": "une_rt_a", 68 "name": "Unemployment rate (annual)", 69 "indicator_code": "UNEMPLOYMENT", 70 "indicator_name": "Unemployment rate (15-74, % active population)", 71 "unit": "%", 72 # une_rt_a の dims: [freq, age, unit, sex, geo, time] 73 "params": {"unit": "PC_ACT", "sex": "T", "age": "Y15-74"}, 74 }, 75 { 76 "code": "tec00118", 77 "name": "HICP - inflation rate (annual)", 78 "indicator_code": "INFLATION", 79 "indicator_name": "HICP inflation rate (all items)", 80 "unit": "%", 81 # tec00118 の dims: [freq, unit, coicop18, geo, time] 82 "params": {"unit": "RCH_A_AVG", "coicop18": "TOTAL"}, 83 }, 84] 85 86 87class EurostatSource(BaseSource): 88 """Eurostat v1 JSON-stat データソース (2026-09-05 実測 200 OK 形式).""" 89 90 source_id = DataSource.EUROSTAT 91 default_url = ( 92 "https://ec.europa.eu/eurostat/api/dissemination/statistics/1.0/data" 93 ) 94 95 def __init__( 96 self, 97 base_url: Optional[str] = None, 98 timeout: float = 30.0, 99 datasets: Optional[list[dict[str, Any]]] = None, 100 countries: Optional[list[dict[str, str]]] = None, 101 ) -> None: 102 """初期化. 103 104 Args: 105 base_url: ベース URL. 106 timeout: タイムアウト秒. 107 datasets: dataset メタ. None なら既定. 108 countries: 国リスト. None なら EU 27 (ISO2). 109 """ 110 super().__init__(base_url=base_url, timeout=timeout) 111 self._datasets = datasets or EUROSTAT_DATASETS 112 self._countries = countries or EUROSTAT_COUNTRIES 113 114 def list_countries(self) -> list[dict[str, Any]]: 115 """国リスト返却.""" 116 return list(self._countries) 117 118 def list_indicators(self) -> list[dict[str, Any]]: 119 """指標メタのリストを返す.""" 120 out: list[dict[str, Any]] = [] 121 for ds in self._datasets: 122 out.append( 123 { 124 "code": ds["indicator_code"], 125 "name": ds["indicator_name"], 126 "unit": ds.get("unit"), 127 "description": ds["name"], 128 } 129 ) 130 return out 131 132 def fetch( 133 self, 134 client: httpx.Client, 135 ) -> Iterable[RawIndicatorRecord]: 136 """v1 JSON-stat API からデータ取得. 137 138 形式: GET /statistics/1.0/data/{DATASET}?format=JSON&geo=XX&... 139 レスポンス: JSON-stat 1.0 (id/size/value 辞書 + dimension.time.category.index) 140 """ 141 country_map = {c["code"]: c for c in self._countries} 142 for ds in self._datasets: 143 dataset_id = str(ds["code"]) 144 for c in self._countries: 145 cc = str(c["code"]) 146 params: dict[str, str] = { 147 "format": "JSON", 148 "geo": cc, 149 "sinceTimePeriod": "2015", 150 } 151 ds_params = ds.get("params") 152 if isinstance(ds_params, dict): 153 params.update({str(k): str(v) for k, v in ds_params.items()}) 154 url = f"{self.default_url}/{dataset_id}" 155 try: 156 resp = client.get(url, params=params) 157 except httpx.HTTPError as exc: 158 logger.warning( 159 "[eurostat] request failed for %s/%s: %s", 160 dataset_id, cc, exc, 161 ) 162 continue 163 if resp.status_code != 200: 164 continue 165 try: 166 payload = resp.json() 167 except Exception as exc: # noqa: BLE001 168 logger.warning("[eurostat] JSON parse failed: %s", exc) 169 continue 170 171 observations = _extract_jsonstat_observations(payload) 172 for year, value in observations: 173 yield RawIndicatorRecord( 174 source=self.source_id, 175 country_code=cc, 176 country_name=country_map.get(cc, {}).get("name", cc), 177 indicator_code=str(ds["indicator_code"]), 178 indicator_name=str(ds["indicator_name"]), 179 year=year, 180 value=safe_float(value), 181 unit=ds.get("unit"), 182 raw_data={"dataset": dataset_id, "country": cc}, 183 ) 184 185 186def _extract_jsonstat_observations( 187 payload: dict[str, Any], 188) -> list[tuple[int, Any]]: 189 """JSON-stat 1.0 から (年, 値) ペアを抽出. 190 191 value のキーは time 次元の index (最後の次元が time の場合)。 192 time category index を参照して実際の期間文字列に変換する。 193 """ 194 obs_pairs: list[tuple[int, Any]] = [] 195 dims = payload.get("id") 196 if not isinstance(dims, list) or not dims: 197 return obs_pairs 198 size = payload.get("size") 199 if not isinstance(size, list): 200 return obs_pairs 201 202 # time 次元の位置とその category index 203 time_pos = dims.index("time") if "time" in dims else len(dims) - 1 204 dimension = payload.get("dimension") 205 time_cat: dict[str, Any] = {} 206 if isinstance(dimension, dict): 207 time_dim = dimension.get("time") 208 if isinstance(time_dim, dict): 209 category = time_dim.get("category") 210 if isinstance(category, dict): 211 idx = category.get("index") 212 if isinstance(idx, dict): 213 time_cat = idx 214 215 # index は {period: position} 形式 216 n_time = size[time_pos] if time_pos < len(size) else 0 217 periods: list[str] = [""] * n_time 218 for period, pos in time_cat.items(): 219 if isinstance(pos, int) and 0 <= pos < n_time: 220 periods[pos] = str(period) 221 222 value = payload.get("value") 223 if not isinstance(value, dict): 224 return obs_pairs 225 226 # 最後の次元が time の場合、value キー = time の位置そのもの 227 last_dim_is_time = time_pos == len(dims) - 1 228 for k, v in value.items(): 229 try: 230 pos = int(k) 231 except (TypeError, ValueError): 232 continue 233 if last_dim_is_time: 234 # 前の次元が 1 値に絞られているため、キーは time index そのもの 235 period = periods[pos] if pos < len(periods) else "" 236 else: 237 # 一般形: キーを多次元 index に変換して time 位置を算出 238 stride = 1 239 strides: list[int] = [] 240 for s in reversed(size): 241 strides.append(stride) 242 stride *= int(s) 243 strides.reverse() 244 if time_pos >= len(strides) or not size[time_pos]: 245 continue 246 time_idx_val = (pos // strides[time_pos]) % int(size[time_pos]) 247 period = periods[time_idx_val] if time_idx_val < len(periods) else "" 248 year_str = period[:4] 249 try: 250 year = int(year_str) 251 except (TypeError, ValueError): 252 continue 253 obs_pairs.append((year, v)) 254 return obs_pairs

1"""World Bank API ソース. 2 3公式: https://api.worldbank.org/v2/ 4全 190+ ヶ国の GDP / 人口 / インフレ / 失業率などを取得できる. 5""" 6from __future__ import annotations 7 8import logging 9from typing import Any, Iterable, Optional 10 11import httpx 12 13from ..models import DataSource, RawIndicatorRecord 14from ..normalizer import safe_float 15from .base import BaseSource 16 17logger = logging.getLogger(__name__) 18 19 20# 代表的指標 (コード → 名前, 単位) 21WORLDBANK_INDICATORS: list[dict[str, Any]] = [ 22 {"code": "NY.GDP.MKTP.CD", "name": "GDP (current US$)", "unit": "USD"}, 23 {"code": "NY.GDP.PCAP.CD", "name": "GDP per capita (current US$)", "unit": "USD"}, 24 {"code": "SP.POP.TOTL", "name": "Population, total", "unit": "persons"}, 25 {"code": "FP.CPI.TOTL.ZG", "name": "Inflation, consumer prices (annual %)", "unit": "%"}, 26 {"code": "SL.UEM.TOTL.ZS", "name": "Unemployment (% of total labor force)", "unit": "%"}, 27 {"code": "SP.DYN.LE00.IN", "name": "Life expectancy at birth (years)", "unit": "years"}, 28 {"code": "NE.RSB.GNFS.CD", "name": "External balance on goods/services (current US$)", 29 "unit": "USD"}, 30] 31 32# 主要国リスト (コード → 名前). World Bank の /country API は 33# 全 217 を返すが、サイズが大きいため MVP では主要国を使う. 34WORLDBANK_COUNTRIES: list[dict[str, str]] = [ 35 {"code": "ARG", "name": "Argentina", "region": "Americas"}, 36 {"code": "AUS", "name": "Australia", "region": "Oceania"}, 37 {"code": "AUT", "name": "Austria", "region": "Europe (EU)"}, 38 {"code": "BEL", "name": "Belgium", "region": "Europe (EU)"}, 39 {"code": "BRA", "name": "Brazil", "region": "Americas"}, 40 {"code": "CAN", "name": "Canada", "region": "Americas"}, 41 {"code": "CHE", "name": "Switzerland", "region": "Europe (EU)"}, 42 {"code": "CHL", "name": "Chile", "region": "Americas"}, 43 {"code": "CHN", "name": "China", "region": "Asia"}, 44 {"code": "COL", "name": "Colombia", "region": "Americas"}, 45 {"code": "CZE", "name": "Czech Republic", "region": "Europe (EU)"}, 46 {"code": "DEU", "name": "Germany", "region": "Europe (EU)"}, 47 {"code": "DNK", "name": "Denmark", "region": "Europe (EU)"}, 48 {"code": "EGY", "name": "Egypt, Arab Rep.", "region": "Africa"}, 49 {"code": "ESP", "name": "Spain", "region": "Europe (EU)"}, 50 {"code": "FIN", "name": "Finland", "region": "Europe (EU)"}, 51 {"code": "FRA", "name": "France", "region": "Europe (EU)"}, 52 {"code": "GBR", "name": "United Kingdom", "region": "Europe (EU)"}, 53 {"code": "GRC", "name": "Greece", "region": "Europe (EU)"}, 54 {"code": "HKG", "name": "Hong Kong SAR, China", "region": "Asia"}, 55 {"code": "HUN", "name": "Hungary", "region": "Europe (EU)"}, 56 {"code": "IDN", "name": "Indonesia", "region": "Asia"}, 57 {"code": "IND", "name": "India", "region": "Asia"}, 58 {"code": "IRL", "name": "Ireland", "region": "Europe (EU)"}, 59 {"code": "ISR", "name": "Israel", "region": "Middle East"}, 60 {"code": "ITA", "name": "Italy", "region": "Europe (EU)"}, 61 {"code": "JPN", "name": "Japan", "region": "Asia"}, 62 {"code": "KOR", "name": "Korea, Rep.", "region": "Asia"}, 63 {"code": "MEX", "name": "Mexico", "region": "Americas"}, 64 {"code": "MYS", "name": "Malaysia", "region": "Asia"}, 65 {"code": "NGA", "name": "Nigeria", "region": "Africa"}, 66 {"code": "NLD", "name": "Netherlands", "region": "Europe (EU)"}, 67 {"code": "NOR", "name": "Norway", "region": "Europe (EU)"}, 68 {"code": "NZL", "name": "New Zealand", "region": "Oceania"}, 69 {"code": "PER", "name": "Peru", "region": "Americas"}, 70 {"code": "PHL", "name": "Philippines", "region": "Asia"}, 71 {"code": "POL", "name": "Poland", "region": "Europe (EU)"}, 72 {"code": "PRT", "name": "Portugal", "region": "Europe (EU)"}, 73 {"code": "ROU", "name": "Romania", "region": "Europe (EU)"}, 74 {"code": "RUS", "name": "Russian Federation", "region": "Europe (EU)"}, 75 {"code": "SAU", "name": "Saudi Arabia", "region": "Middle East"}, 76 {"code": "SGP", "name": "Singapore", "region": "Asia"}, 77 {"code": "SWE", "name": "Sweden", "region": "Europe (EU)"}, 78 {"code": "THA", "name": "Thailand", "region": "Asia"}, 79 {"code": "TUR", "name": "Türkiye", "region": "Middle East"}, 80 {"code": "TWN", "name": "Taiwan, China", "region": "Asia"}, 81 {"code": "UKR", "name": "Ukraine", "region": "Europe (EU)"}, 82 {"code": "USA", "name": "United States", "region": "Americas"}, 83 {"code": "VNM", "name": "Vietnam", "region": "Asia"}, 84 {"code": "ZAF", "name": "South Africa", "region": "Africa"}, 85] 86 87 88class WorldBankSource(BaseSource): 89 """World Bank v2 API データソース.""" 90 91 source_id = DataSource.WORLDBANK 92 default_url = "https://api.worldbank.org/v2" 93 94 def __init__( 95 self, 96 base_url: Optional[str] = None, 97 timeout: float = 30.0, 98 indicators: Optional[list[dict[str, Any]]] = None, 99 countries: Optional[list[dict[str, str]]] = None, 100 ) -> None: 101 """初期化. 102 103 Args: 104 base_url: API ベース URL. 105 timeout: HTTP タイムアウト. 106 indicators: 取得対象指標. None なら既定セット. 107 countries: 取得対象国. None なら既定セット. 108 """ 109 super().__init__(base_url=base_url, timeout=timeout) 110 self._indicators = indicators or WORLDBANK_INDICATORS 111 self._countries = countries or WORLDBANK_COUNTRIES 112 113 def list_countries(self) -> list[dict[str, Any]]: 114 """国が事前定義されているのでそのリストを返す.""" 115 return list(self._countries) 116 117 def list_indicators(self) -> list[dict[str, Any]]: 118 """指標メタのリストを返す.""" 119 return list(self._indicators) 120 121 def fetch( 122 self, 123 client: httpx.Client, 124 ) -> Iterable[RawIndicatorRecord]: 125 """World Bank API から指標データを取得して yield. 126 127 形式: GET /v2/country/{CCs}/indicator/{IND}?format=json&per_page=1000 128 注意: indicator は 1 リクエスト 1 個のみ (セミコロン連結は非対応). 129 130 Yields: 131 RawIndicatorRecord. 132 """ 133 country_codes = ";".join(c["code"] for c in self._countries) 134 ind_map = {i["code"]: i for i in self._indicators} 135 country_map = {c["code"]: c for c in self._countries} 136 137 for ind in self._indicators: 138 ind_code = ind["code"] 139 url = f"{self.base_url}/country/{country_codes}/indicator/{ind_code}" 140 params: dict[str, Any] = { 141 "format": "json", 142 "per_page": 20000, 143 "date": "2015:2024", 144 } 145 try: 146 resp = client.get(url, params=params) 147 except httpx.HTTPError as exc: 148 logger.warning( 149 "[worldbank] request failed (%s): %s", ind_code, exc) 150 continue 151 if resp.status_code != 200: 152 logger.warning( 153 "[worldbank] non-200 status (%s): %d %s", 154 ind_code, resp.status_code, resp.text[:200], 155 ) 156 continue 157 158 try: 159 payload = resp.json() 160 except Exception as exc: # noqa: BLE001 161 logger.warning( 162 "[worldbank] JSON parse failed (%s): %s", ind_code, exc) 163 continue 164 165 # 形式: [metadata, data] の 2 要素 166 if not isinstance(payload, list) or len(payload) < 2: 167 logger.warning( 168 "[worldbank] unexpected payload format (%s)", ind_code) 169 continue 170 rows = payload[1] 171 if not isinstance(rows, list): 172 continue 173 174 for row in rows: 175 if not isinstance(row, dict): 176 continue 177 cc = row.get("countryiso3code") or "" 178 ind_id = ind_code 179 value = safe_float(row.get("value")) 180 year_raw = row.get("date") 181 try: 182 year = int(str(year_raw)) 183 except (TypeError, ValueError): 184 continue 185 if not cc or not ind_id: 186 continue 187 188 ind_meta = ind_map.get(ind_id, {}) 189 c_meta = country_map.get(cc, {}) 190 country_value = row.get("country") 191 yield RawIndicatorRecord( 192 source=self.source_id, 193 country_code=cc, 194 country_name=( 195 c_meta.get("name") 196 or (country_value.get("value", "") 197 if isinstance(country_value, dict) else "") 198 ), 199 indicator_code=ind_id, 200 indicator_name=ind_meta.get("name") 201 or (ind.get("value", "") 202 if isinstance(ind, dict) else ""), 203 year=year, 204 value=value, 205 unit=ind_meta.get("unit"), 206 raw_data=row, 207 )