US Hazardous Freight Flow Benchmarks — Lane Records, Per Record

Turn the U.S. Commodity Flow Survey hazmat file into clean, per-record freight-lane benchmarks — one row per origin x destination x commodity x hazmat class x mode x year, with shipped tons, value and average miles, ready to size hazardous-goods lanes across the U.S.

Each source row becomes one clean, flat record with numeric fields coerced to real numbers, a stable source-native record_id, and provenance stamped on every row: source, resource id, the licence notice, the required attribution, a UTC retrieval timestamp and an interpretation caveat.

What one record represents

The source is dkgi-gbeh — CFS Hazmat File 2012-2022 (Commodity Flow Survey) on the U.S. DOT open-data portal (data.bts.gov). Each record is one hazardous-goods freight lane: an origin geography and a domestic or export destination, a commodity (SCTG) and NAICS industry, a UN/NA hazardous-material number and hazard class, a transport mode and a survey year, together with the shipped tons, value and average miles. Origin and destination are geographies, not persons.

For each record you get a composite record_id built from the source-native key, the analytic columns listed below (reproduced verbatim, numbers as numbers), and the provenance block. Columns include origin (geotype, st, geo_id, geo_ttl), destination (ddestgeo/ddestgeo_ttl, xdestgeo/xdestgeo_ttl), commodity (naics/naics_ttl, comm/comm_ttl), hazmat (unna/unna_ttl, hazclass/hazclass_ttl, hazstat/hazstat_ttl), dmode/dmode_ttl (mode), year, and the metrics ton, val and avgmile.

Coverage and volume

The live file holds 40,443 hazardous-freight lane records spanning the 2012-2022 Commodity Flow Survey hazmat detail — that is the record capacity of a full pull.

Live count: 40,443 records — matches the Wave-3 index figure exactly.

The Actor pages the source with keyless SODA $query requests ordered by the source-native key for a stable total order, and stops as soon as your Maximum records cap is met. The Actor names every wanted column explicitly in a SoQL $query so no column hidden by a default view is lost.

Licence and attribution

This is a public-domain U.S. Government work (17 U.S.C. §105). The full notice travels on every record:

U.S. DOT / Bureau of Transportation Statistics. Public-domain U.S. Government work (17 U.S.C. 105). Reproduced verbatim; no third-party content or logos.

The required attribution — U.S. DOT / Bureau of Transportation Statistics (Commodity Flow Survey) — travels on every record.

Interpretation caveat

Each record is a Commodity Flow Survey estimate for a hazardous-goods lane, not a shipment-level record. Tons and value are survey totals; average miles is per shipment. Some rows are national or all-commodity aggregates - read the geography and commodity titles.

Values are reproduced verbatim: the Actor never rescales, re-derives or editorialises a number.

Person-data policy

Origin and destination are geographies and commodities are goods classes - there are no name, email, phone, personal-address or personal-identifier fields. A per-record assertion enforces the person-field allow-list at write time.

Data quality and freshness

Numeric fields are coerced from the source's string encoding into real numbers (integers where whole, floats otherwise); genuinely missing cells are delivered as null, never as zero. Text is passed through verbatim. Every run re-reads the live source, so the data is as fresh as the portal itself, and each record's observed_at stamp records exactly when the row was retrieved. Delivery order is fixed by the source-native key, so a capped sample and a later full pull agree on their overlap and a repeated run returns rows in the same order. The RUN_RECEIPT reports source rows scanned and records delivered and charged for a per-run reconciliation.

Provenance, licensing and compliance

Every run begins with a live source-preflight: the Actor reads the exact host's robots.txt at runtime and refuses to proceed if the crawl policy disallows the data path. The gate result — URL, HTTP status, byte length and a SHA-256 of the policy — is written to the run's RUN_RECEIPT, so each run carries its own audit trail. The Actor identifies itself with a transparent, non-impersonating User-Agent and never bypasses a block, solves a challenge, or fetches through a cache or mirror. When the door is genuinely unavailable the run fails loudly and bills nothing.

Inputs

• Maximum records (maxRecords) — hard cap on records delivered and billed. Raise it to pull the full set; lower it to sample cheaply. Records arrive in a stable, source-native order.

Output

Records land in the Actor's default dataset and export as JSON, CSV, Excel or via the Apify API. A tabular overview view surfaces the most useful columns for quick inspection while the full record retains every selected field and provenance stamp.

Fields in detail

The record leads with record_id — a stable composite key drawn from the source's own grain — followed by the analytic columns described above and closed by a provenance block: source, source_dataset (the Socrata resource id), licence, attribution, caveat and observed_at. Every one of those provenance fields is present on every record, so a single row is self-describing: hand it to a colleague or a downstream system and it carries its own origin, licence and retrieval time without reference back to this page. Because delivery is ordered by the source-native key, the same record always carries the same record_id across runs, which makes the dataset safe to diff, deduplicate, or upsert into a warehouse. Nothing in the record is computed or inferred beyond the explicit count where one is stated — every other value is the source's own, reproduced byte-for-byte.

Sibling Actors

This Actor sizes bulk hazardous-freight lanes in one pull; it complements the owned Supply Chain MCP tool, which answers per-call supply-chain questions interactively. Bulk lane benchmarks versus an interactive tool - two different jobs. This Actor also shares its engineering — the runtime robots gate, push-then-charge billing and verbatim-value discipline — with the fleet's other public-data records Actors.

Pricing

This Actor uses Apify's pay-per-event model: a flat $0.05 per record actually delivered to the dataset, and nothing else — no monthly rental, no per-run base fee, no compute charge. Deliver 40 records and you pay $2.00; deliver 10,000 and you pay $500.00. Billing is wired after delivery — each record is pushed first and only then does the per-record event fire — so a mid-run failure can only ever under-charge you, never over-charge. Use Maximum records to cap spend precisely.

Scaling and limits

Set Maximum records low to sample the leading slice cheaply, or high to pull the full set. The Actor paginates server-side and delivers incrementally, so memory stays flat regardless of how many records you request, and you are billed only for what is actually delivered. Because the source is a live public API, extremely deep pagination is ultimately bounded by the source's own paging behaviour; for the vast majority of uses — sampling, a full refresh, or a scheduled top-up — the default paging is more than sufficient. Schedule the Actor on Apify to keep a downstream table current: each run re-reads the live source and re-stamps observed_at, so a nightly or weekly run gives you a dated, reproducible snapshot.

Typical uses

Size hazardous-goods lanes by origin, destination, commodity and mode; rank lanes by tons or value; screen where a hazard class moves and how far; benchmark modal split for hazmat; or feed a logistics-risk, routing or insurance model with clean lane records.

What this Actor does not do

It does not forecast or model, does not merge multiple source tables into one record, and it contains no personal data - only geographies, commodities and freight metrics. It gives you faithful, analysis-ready records — with a provenance trail you can audit on every run.