# EIA Scraper - US Energy Data (Petroleum, Gas, Electricity) (`crawlerbros/eia-scraper`) Actor Scrape U.S. Energy Information Administration (EIA) open data - energy prices, production, consumption, and stocks across petroleum, natural gas, electricity, coal, nuclear, and total energy, by state and time period. No API key required. - **URL**: https://apify.com/crawlerbros/eia-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Automation, Developer tools, Integrations - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## EIA Scraper — US & Global Energy Data (Petroleum, Gas, Electricity, Coal, Grid Ops) Scrape official U.S. Energy Information Administration (EIA) open data — energy prices, production, consumption, stocks, imports, grid operations, and outlook forecasts across **petroleum**, **natural gas**, **electricity**, **coal**, **nuclear**, **total energy**, **international energy**, and more, broken down by US state, country, and time period. No API key, no login, no proxy required. ### What this actor does - **12 energy categories:** Natural Gas, Petroleum, Coal, Electricity, Total Energy, Nuclear Outages, Short-Term Energy Outlook, State Energy Data System, International Energy, Petroleum Imports, Annual Energy Outlook, Electricity Grid Operations - **Filter by keyword, exact series ID, frequency (including hourly grid data), US state, and year range** - **Flat, analysis-ready rows:** one row per (series, time period) data point - **Empty fields are omitted** — no null clutter in your dataset ### Output per data point - `category` — energy category label (e.g. `Petroleum`) - `seriesId` — EIA series identifier (e.g. `PET.RWTC.D`) - `name`, `description` — human-readable series description - `units` — unit of measure (e.g. `Dollars per Gallon`, `Thousand Barrels per Day`, `Megawatthours`) - `frequency` — `Annual` / `Quarterly` / `Monthly` / `Weekly` / `Daily` / `Hourly (UTC)` / `Hourly (Local Time)` / `4-Week Average` - `geography` — EIA geography code (e.g. `USA-TX` for a US state, `ARM` for Armenia, `WLD` for World) when the series is state/country-scoped - `geographyDestination` — for Petroleum Imports, the US destination state/port (e.g. `USA-MS`) when `geography` holds the origin country - `latitude`, `longitude` — decimal coordinates of the import terminal, populated only for Petroleum Imports series that publish them - `period` — raw EIA period string (e.g. `2024`, `2024Q1`, `202401`, `20240115`, `20260717T06`) - `periodYear` — the 4-digit year extracted from `period` - `value` — the numeric data value - `dataSource` — the underlying data provider EIA credits for this series (e.g. `Thomson-Reuters` for WTI spot prices, `EIA, U.S. Energy Information Administration` for most others), when published - `lastUpdated` — when EIA last refreshed this series - `sourceUrl` — link to EIA's public query-browser page for this series - `recordType: "energyDataPoint"`, `scrapedAt` ### Input | Field | Type | Default | Description | |---|---|---|---| | `category` | select | `naturalGas` | Which EIA dataset to query | | `keyword` | string | – | Free-text match against series name/description/ID | | `seriesId` | string | – | Exact EIA series ID lookup (auto-detects category) | | `frequency` | select | `any` | `annual` / `quarterly` / `monthly` / `weekly` / `daily` / `hourly` / `hourlyLocal` / `fourWeekAvg` | | `region` | select | `US` | Restrict to one US state (or `US` for no filter); for Petroleum Imports, matches the US destination state | | `startYear` | int | – | Only include data from this year onward | | `endYear` | int | – | Only include data up to this year | | `maxItems` | int | `100` | Hard cap on emitted rows (1–5000) | #### Example: WTI crude oil daily spot price ```json { "category": "petroleum", "seriesId": "PET.RWTC.D", "maxItems": 500 } ```` #### Example: Weekly natural gas storage, keyword search ```json { "category": "naturalGas", "keyword": "working underground storage", "frequency": "weekly", "maxItems": 200 } ``` #### Example: California electricity retail sales, 2020–2024 ```json { "category": "electricity", "keyword": "retail sales", "region": "CA", "startYear": 2020, "endYear": 2024 } ``` #### Example: US state-level total energy prices ```json { "category": "stateEnergyData", "keyword": "average price", "region": "TX", "frequency": "annual" } ``` #### Example: International crude oil production by country ```json { "category": "internationalEnergy", "keyword": "Armenia", "maxItems": 100 } ``` #### Example: Crude oil imports into a US state, by origin country ```json { "category": "petroleumImports", "region": "MS", "startYear": 2024 } ``` #### Example: Long-term US electricity price outlook ```json { "category": "annualEnergyOutlook", "keyword": "electricity", "maxItems": 200 } ``` #### Example: Hourly grid generation by fuel type ```json { "category": "electricityGridOperations", "keyword": "solar", "frequency": "hourly", "maxItems": 500 } ``` ### Use cases - **Energy market research** — track historical crude oil, gasoline, diesel, and natural gas prices - **Utility & grid analytics** — electricity generation, sales, and customer counts by US state - **ESG / sustainability reporting** — total-energy consumption trends by sector - **Academic & policy research** — bulk time-series export for econometric modeling - **Trading & forecasting** — Short-Term Energy Outlook (STEO) and Annual Energy Outlook (AEO) forward-looking series - **Nuclear reliability monitoring** — reactor-level outage and capacity data - **Global energy comparisons** — country-level production/consumption for 200+ countries and regions - **Trade-flow analysis** — crude oil import volumes by origin country and US port/refinery - **Grid & renewables monitoring** — hourly demand, generation-by-fuel, and interchange for every US balancing authority ### Data Source This actor reads EIA's free, keyless **bulk-download files** (published at [eia.gov/opendata/bulkfiles.php](https://www.eia.gov/opendata/bulkfiles.php)) rather than EIA's `api.eia.gov` v2 REST API. Here's why: - `api.eia.gov/v2/...` **hard-requires** a registered `api_key` query parameter. A request with no key, or an empty key, returns `HTTP 403 {"error":{"code":"API_KEY_MISSING", ...}}`. Free self-serve registration exists, but per this project's zero-cost policy an actor must not require end users to obtain and paste in their own API key. - The API is hosted on the API Umbrella gateway shared with `api.data.gov` (confirmed via response headers), so its documented shared testing credential `DEMO_KEY` does work — but it is capped at a very low shared rate limit (observed `X-Ratelimit-Limit: 10` per window, shared globally across *every* consumer of every federal API on that gateway). That's unusable for a reliable production scraper and isn't something EIA documents as intended for this purpose, so this actor does not hardcode it. - EIA's **bulk ZIP files** (`NG.zip`, `PET.zip`, `COAL.zip`, `ELEC.zip`, `TOTAL.zip`, `NUC_STATUS.zip`, `STEO.zip`, `SEDS.zip`, `INTL.zip`, `PET_IMPORTS.zip`, `AEO2026.zip`, `EBA.zip`) require **zero credentials**, are served directly from `eia.gov` with no anti-bot gate, and contain the exact same underlying series data (in fact the same `series_id`/`data[period, value]` records) that the v2 API would return. This actor downloads the relevant bulk file per request and filters it in-memory to match your query. The trade-off: the `electricityGridOperations` and `electricity` categories' bulk files are large (~660 MB and ~240 MB compressed respectively), so queries against them take noticeably longer than other categories. All other categories are small (a few MB to ~60 MB) and resolve quickly. ### FAQ **Do I need an EIA API key?** No. This actor uses EIA's public bulk-download files, which require no registration, login, or key. **How current is the data?** EIA refreshes its bulk files on the same cadence as the source surveys — daily for petroleum/natural gas prices, weekly for storage, monthly/quarterly/annual for most other series. Each record's `lastUpdated` field reflects EIA's own refresh timestamp. **What's a "series"?** EIA organizes data into time series — a single measured quantity over time (e.g. "Cushing, OK WTI Spot Price FOB, Daily"). Each series has a unique `seriesId`. This actor flattens each series' historical points into individual rows. **How do I find a series ID?** Set `keyword` to a descriptive term (e.g. `"WTI spot price"`, `"retail gasoline"`, `"natural gas storage"`) and browse the returned `seriesId` values, or browse EIA's own [Open Data query browser](https://www.eia.gov/opendata/qb.php). **Why is `geography` sometimes missing?** Not every EIA category is geography-scoped. National/aggregate series (Short-Term Energy Outlook country-level forecasts, most Natural Gas series) omit it; state-level categories (Electricity, State Energy Data System, some Petroleum PADD series) populate it. **Why do the `electricity` and `electricityGridOperations` categories take longer to run?** Their bulk files are the largest EIA publishes (~240 MB and ~660 MB respectively). All other categories are much smaller and typically finish in well under a minute. **Can I filter by PADD region (petroleum)?** PADD regions aren't exposed as a dedicated dropdown (EIA encodes them inside series descriptions, e.g. "Gulf Coast (PADD 3)"), but `keyword: "PADD 3"` or `keyword: "Gulf Coast"` reliably finds them. **Can I filter International Energy data by country?** Country isn't a separate dropdown (EIA covers 200+ countries and regions), but every series name embeds the country (e.g. "Petroleum and other liquids production, Armenia, Quarterly"), so `keyword: "Armenia"` reliably finds it. **What's the difference between `geography` and `geographyDestination` on Petroleum Imports rows?** `geography` is the crude oil's origin (a country code, or `WLD` for "World"); `geographyDestination` is the receiving US state. Use the `region` filter to restrict by destination state. **Which Annual Energy Outlook edition does this actor use?** The latest published edition (AEO2026), which projects US energy trends through 2050. **Is this official EIA data?** Yes — every record comes directly from EIA's own published bulk files, unmodified beyond flattening and field renaming for consistency. # Actor input Schema ## `category` (type: `string`): Which EIA energy dataset to query. ## `keyword` (type: `string`): Free-text filter matched against the series name, description, and ID (e.g. `diesel`, `storage`, `generation`, `retail sales`). ## `seriesId` (type: `string`): Look up one exact EIA series ID (e.g. `PET.RWTC.D` for WTI crude spot price). If set, this overrides other filters and auto-detects the correct category from the ID's prefix. ## `frequency` (type: `string`): Restrict to series reported at this frequency. ## `region` (type: `string`): Restrict to series covering this US state. For Petroleum Imports, matches the US destination port's state. National/all-US series and categories without state-level breakdowns are unaffected by this filter (e.g. International Energy is filtered by country name via Keyword instead). ## `startYear` (type: `integer`): Only include data points from this year onward. ## `endYear` (type: `integer`): Only include data points up to and including this year. ## `maxItems` (type: `integer`): Hard cap on emitted data-point records. ## Actor input object example ```json { "category": "naturalGas", "frequency": "any", "region": "US", "maxItems": 100 } ``` # Actor output Schema ## `energyData` (type: `string`): Dataset containing all scraped EIA energy data points. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "category": "naturalGas", "frequency": "any", "region": "US", "maxItems": 100 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/eia-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "category": "naturalGas", "frequency": "any", "region": "US", "maxItems": 100, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/eia-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "category": "naturalGas", "frequency": "any", "region": "US", "maxItems": 100 }' | apify call crawlerbros/eia-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/eia-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "EIA Scraper - US Energy Data (Petroleum, Gas, Electricity)", "description": "Scrape U.S. Energy Information Administration (EIA) open data - energy prices, production, consumption, and stocks across petroleum, natural gas, electricity, coal, nuclear, and total energy, by state and time period. No API key required.", "version": "1.0", "x-build-id": "ZsMicwORzw2Uuty7v" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~eia-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-eia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~eia-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-eia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~eia-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-eia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "category" ], "properties": { "category": { "title": "Energy category", "enum": [ "naturalGas", "petroleum", "coal", "electricity", "totalEnergy", "nuclearOutages", "shortTermOutlook", "stateEnergyData", "internationalEnergy", "petroleumImports", "annualEnergyOutlook", "electricityGridOperations" ], "type": "string", "description": "Which EIA energy dataset to query.", "default": "naturalGas" }, "keyword": { "title": "Keyword", "type": "string", "description": "Free-text filter matched against the series name, description, and ID (e.g. `diesel`, `storage`, `generation`, `retail sales`)." }, "seriesId": { "title": "Exact series ID", "type": "string", "description": "Look up one exact EIA series ID (e.g. `PET.RWTC.D` for WTI crude spot price). If set, this overrides other filters and auto-detects the correct category from the ID's prefix." }, "frequency": { "title": "Frequency", "enum": [ "any", "annual", "quarterly", "monthly", "weekly", "daily", "hourly", "hourlyLocal", "fourWeekAvg" ], "type": "string", "description": "Restrict to series reported at this frequency.", "default": "any" }, "region": { "title": "US state / region", "enum": [ "US", "AL", "AK", "AZ", "AR", "CA", "CO", "CT", "DE", "DC", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY" ], "type": "string", "description": "Restrict to series covering this US state. For Petroleum Imports, matches the US destination port's state. National/all-US series and categories without state-level breakdowns are unaffected by this filter (e.g. International Energy is filtered by country name via Keyword instead).", "default": "US" }, "startYear": { "title": "Start year", "minimum": 1900, "maximum": 2100, "type": "integer", "description": "Only include data points from this year onward." }, "endYear": { "title": "End year", "minimum": 1900, "maximum": 2100, "type": "integer", "description": "Only include data points up to and including this year." }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Hard cap on emitted data-point records.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```