KAPSARC Energy Data Scraper

🚀 Export KAPSARC energy datasets in seconds. Production, consumption, trade flows, and price indicators from KAPSARC public catalog.

🕒 Last updated: 2026-06-05 . 📊 6 fields per record . Public API . Real-time

The KAPSARC Energy Data Scraper turns the datasource.kapsarc.org public REST endpoint into a clean, structured dataset.

🎯 Target Audience	💡 Primary Use Cases
📊 Energy analysts	Pull oil and gas indicators
💼 Policy teams	Snapshot energy trade flows
🤖 ML engineers	Build training sets on energy data
📰 Journalists	Verify policy figures

📋 What the KAPSARC Energy Data Scraper does

• Calls https://datasource.kapsarc.org/api/records/1.0/search with the dataset you supply.
• Flattens fields per record into a clean row.
• Supports text query, refine filters, and sort.
• Surfaces upstream errors as a clean error record.

💡 Why it matters. The source returns raw API payloads that most data tools cannot ingest directly. This actor normalizes everything into a flat row format ready for analysis.

🎬 Full Demo

🚧 Coming soon.

⚙️ Input

Field	Type	Required	Description
maxItems	integer	No	Free users limited to 10. Paid users up to 1,000,000. Prefill 10.

Example 1.

{
  "dataset": "india-crude-oil-indicators",
  "maxItems": 100
}

Example 2.

{
  "dataset": "india-crude-oil-indicators",
  "q": "refinery",
  "maxItems": 50
}

⚠️ Good to Know. This actor uses a public endpoint. No login or API key is required.

📊 Output

Field	Type	Description
📁 datasetid	string	KAPSARC dataset id.
🆔 recordid	string	Unique record id.
🕒 recordTimestamp	string	Record timestamp from upstream.
📊 fields	object	Flattened upstream fields.
🕒 scrapedAt	string	When this row was fetched.
❌ error	string	Set if upstream returned an error.

Sample record.

{
  "datasetid": "india-crude-oil-indicators",
  "recordid": "abc123",
  "recordTimestamp": "2025-11-09T10:24:58.941Z",
  "fields": {
    "obs_value": "0.0",
    "energy_product_name": "Total",
    "flow_breakdown": "Refinery intake",
    "time_period": "2016-11-01",
    "unit_measure": "CONVBBL"
  },
  "scrapedAt": "2026-06-05T12:00:00.000Z",
  "error": null
}

✨ Why choose this Actor

| 🆓 | Public source, no API key required. | | 🧹 | Clean snake_case fields ready for BI tools. | | 🔢 | Numeric values auto-cast for spreadsheets. | | 🛟 | Errors surfaced as a clean record instead of crashing. | | 💾 | Push to dataset for instant export in multiple formats. |

📈 How it compares to alternatives

Approach	Setup time	Clean fields?	Numeric casting?
Roll your own fetch	30 min+	❌	❌
Custom Python script	1 hr+	partial	partial
This Actor	5 sec, no install	✅	✅

🚀 How to use

1. Click Try for free.
2. Adjust maxItems if needed (defaults to 10).
3. Click Start. Within seconds, your dataset is ready for download or integration.

💼 Business use cases

📊 Analytics dashboards. Pipe results into BI tools.

💼 Backtesting. Snapshot data daily for reproducible analysis.

📰 Reporting. Embed live numbers in newsletters or briefings.

🤖 ML feature engineering. Build training sets from public data.

🔌 Automating KAPSARC Energy Data Scraper

• Make / Zapier. Trigger this actor on a schedule, push results to Airtable, Google Sheets, or Slack.
• Cron schedule. Native Apify scheduler.
• Webhooks. POST to your endpoint when a run finishes.
• Pipe to BigQuery, Snowflake, Postgres. Native Apify integrations.

🌟 Beyond business use cases

🎓 Education. Use real data in finance and economics classes.

🧪 Personal research. Track your own metrics without coding.

🤝 Non-profit and open data. Build public dashboards with current numbers.

🧰 Tinkering and prototyping. Spin up a feed in seconds for new ideas.

🤖 Ask an AI assistant about this scraper

Pop this README into ChatGPT, Claude, or any AI assistant and ask it to map your workflow to the actor inputs.

❓ Frequently Asked Questions

❓ Do I need an API key? No. KAPSARC datasource is fully public.

❓ Where do I find dataset ids? Browse the KAPSARC catalog at datasource.kapsarc.org/explore/.

❓ Can I filter by year or country? Yes, use the refineKey and refineValue inputs.

❓ Will the schema change? datasetid, recordid, recordTimestamp, and fields are stable.

❓ Can I schedule runs? Yes via Apify scheduler, Make, Zapier, or cron.

❓ Is this scraping or API? API. KAPSARC search endpoint is public and stable.

❓ What if upstream returns an error? A single record with error populated is pushed.

❓ Can I run heavy backfills? Yes on a paid plan up to 1,000,000 items.

❓ Which export formats are available? Apify dataset UI exposes multiple structured formats.

❓ Is rate limiting strict? KAPSARC is generous with public traffic.

🔌 Integrate with any app

Apify ships native integrations with Make, Zapier, Slack, Discord, Google Drive, Google Sheets, Gmail, Airbyte, Keboola, Telegram, GitHub, and any REST API or webhook endpoint.

🔗 Recommended Actors

Actor	What it does
ParseForge Alpha Vantage Scraper	Stocks, FX, crypto, and indicators.
ParseForge OurAirports Scraper	Global airport database.
ParseForge NBA Stats Scraper	Player and team stats from NBA.com.
ParseForge CurseForge Mods Scraper	Public mod metadata from CurseForge.

💡 Pro Tip. Browse the complete ParseForge collection for 900+ production-grade scrapers.

---

Disclaimer. This actor scrapes only publicly available data. ParseForge is not affiliated with, endorsed by, or sponsored by any of the third-party services referenced. Users are responsible for complying with the target site terms of service and applicable law. Create a free account w/ $5 credit.