AEMO NEM Price and Demand CSV Backfill

Download the Australian Energy Market Operator's official monthly price and demand CSV files for the National Electricity Market and get one JSON row per five-minute source label. Each row keeps the source's own SETTLEMENTDATE string and its RRP and TOTALDEMAND values exactly as published, with the file's URL, SHA-256 and fetch time. A free SUMMARY record reports coverage (expected, present and missing labels), provenance and every problem met during the run.

This is a raw-source backfill: it does not convert timestamps to UTC, does not say whether a label marks the start or end of an interval, and does not attach units or currency to the values. Apply your own definitions downstream. Region NSW1 is the only region enabled in this version; months from October 2025 onward can be requested.

Typical use: import one to three months into your own store, then schedule a nightly run for the current or previous month and upsert by the row id.

Input

{
    "regions": ["NSW1"],
    "startMonth": "2026-09",
    "endMonth": "2026-09",
    "maxRows": 10000,
    "requestIntervalMs": 1100
}

• regions: NSW1 only for now. QLD1, SA1, TAS1 and VIC1 are rejected (region_not_enabled) before any request until each has a verified monthly file.
• startMonth / endMonth: YYYY-MM, inclusive, 2025-10 or later, at most three consecutive months, not after the current UTC month. Give both or neither; leaving both empty selects the previous UTC calendar month (a scheduling convention only). The current UTC month is allowed and is always reported as open_month.
• maxRows: 1 to 100,000 rows stored across all files (default 10,000). A full 30-day month has 8,640 rows and a 31-day month 8,928, so the default covers one month.
• requestIntervalMs: idle wait after each response, 1,100 to 10,000 ms. One request is ever in flight.

Unknown fields, credentials, proxies and custom headers are rejected.

Output

One dataset row per unique five-minute label in the requested files, ordered by month, region and label:

{
    "schemaVersion": "raw-1",
    "id": "aemo-price-demand-raw-v1:2026-09:NSW1:20260901T000500:TRADE",
    "region": "NSW1",
    "sourceSettlementDate": "2026/09/01 00:05:00",
    "sourceRrp": "64.95",
    "sourceTotalDemand": "7694.77",
    "periodType": "TRADE",
    "sourceMonth": "2026-09",
    "sourceUrl": "https://www.aemo.com.au/aemo/data/nem/priceanddemand/PRICE_AND_DEMAND_202609_NSW1.csv",
    "sourceSha256": "85af3e1f68823c3eef87fcc98e5fa605c609b568ec64d6ef36b96722f8db8cd0",
    "observedAt": "2026-10-17T03:21:28.412Z"
}

• sourceSettlementDate is the unzoned label as published. Each monthly file runs from the first label after 00:00 on the 1st to 00:00 on the 1st of the next month, so September's last row is 2026/10/01 00:00:00.
• sourceRrp and sourceTotalDemand are strings, so spelling and precision are preserved. Negative prices are kept as they are.
• id is stable for the same file month, region, label and period type. A revised value keeps its id and changes the value string; it is not a global identity across other AEMO datasets.
• observedAt is when the file was fetched, not when AEMO revised it.

Each file is validated before any of its rows are stored: exact header, region, partition, five-minute grid, decimal format and period type. A file that fails is reported in SUMMARY and contributes no rows. Identical duplicate lines collapse; conflicting duplicates fail the file. Gaps are reported as missing ranges and never turned into deletion rows.

SUMMARY (free, default key-value store)

SUMMARY holds the run status (complete, partial, capped or failed), the normalized input, request and byte counts, parsed and stored row counts, whether maxRows or your spending limit stopped the run, an errors list and one record per requested file: URL, HTTP status, fetch time, SHA-256, source/unique/duplicate rows, expected and missing labels with up to 100 missing ranges, first and last label, coverage status (complete, incomplete, open_month, unavailable), stored rows and whether delivery finished. A missing or blocked file is unavailable, never zero demand. sourceSemantics states that time basis, timestamp meaning and units are unspecified.

After a restart of the same run, existing dataset rows count toward both row and spending limits. Their IDs are checked before more delivery; unreadable or inconsistent storage stops the run. Already stored IDs are skipped when a file is fetched again. Historical charges remain null when they cannot be confirmed. Before a file is revalidated, its coverage is unavailable and parsedRows includes only the restored rows as a lower bound. A block or other stop raised before the restart still holds, and the HTTP attempt and byte budgets continue from where they were: both live in the free SOURCE_STATE record of the default key-value store, written before every request. A request whose outcome was never recorded (the run restarted while it was in flight and the result could not be written) stops further AEMO requests. Separate new runs still deliver and bill complete snapshots again.

Pricing

Pay per event:

• Actor start: $0.00005 per run.
• Price and demand row: $0.0002 per row stored in the dataset.

You pay for every row delivered, including rows that are unchanged since an earlier run: a full 30-day month (8,640 rows) costs about $1.73, and refreshing the same month daily for 30 days costs about $51.84. There is no changes-only mode. SUMMARY, error reports and missing files are free, but a failed run still incurs the start event and platform usage. Rows stop at maxRows or your spending limit, whichever comes first; a capped snapshot is marked incomplete and you can narrow the next request or raise the cap.

Limits and behaviour

• At most 3 months and 15 files per run, 48 HTTP attempts, a 1 MiB body cap per file, 32 MiB of responses and a 600-second budget for all work.
• Server errors and connection resets are retried at most twice (2 s, then 5 s, or a longer Retry-After up to 60 s). A missing file (404/410) is reported as source_unavailable and the run moves on.
• Blocks stop the run. AEMO's site sits behind Cloudflare. A 403, a 429, a challenge page on any HTTP status or a redirect to a challenge handler stops all remaining AEMO requests for the run, with no retry and no alternate host, browser or proxy. Rows already stored stay in the dataset and SUMMARY says what happened. Access from Apify's cloud network is not yet proven.
• Only https://www.aemo.com.au and the fixed monthly CSV route are requested; redirects anywhere else are refused.

Source and data notes

Data comes from AEMO's public aggregated price and demand files (https://www.aemo.com.au/aemo/data/nem/priceanddemand/PRICE_AND_DEMAND_{YYYYMM}_{REGION}.csv), linked from AEMO's aggregated data page. AEMO is the data controller; check its copyright and data terms before redistribution. The files contain no personal data. This actor is not affiliated with AEMO.

Support

Open an issue on the actor's Issues tab with your run id and the SUMMARY record.