uData Portal Extractor

Point at any uData open-data portal and get clean, structured rows. uData (built by Etalab) is the platform behind data.gouv.fr and roughly 30 francophone / EU government open-data portals, and every deployment exposes the same uniform /api/1/… REST API — so this is one generic runner, not a per-portal scraper. Give it a portal base URL and a mode.

uData is distinct from CKAN, Socrata, and OpenDataSoft. Its /api/1/ shape, its 1-indexed page / page_size pagination, its nested per-dataset resources[], and its community reuses graph are uData-specific.

Modes

mode	Endpoint	One row per…
datasets (default)	/api/1/datasets/	dataset — with its resources[] summarised on (resourceCount, first-resource url/format/filesize/mime) and the full lossless resources array kept
organizations	/api/1/organizations/	publishing organisation (name, website, dataset/reuse/member counts)
reuses	/api/1/reuses/	community reuse — an app / article / visualisation built on the portal's data (a uData-unique layer)

With emitResources on (datasets mode), one extra row per resource — each an actual downloadable file (url / format / filesize / checksum / mime) — is also emitted, joined to its parent dataset.

Facet filters

All passed straight through to the uData API and combinable:

• query — full-text q= filter
• tag — tag facet
• organization — an organization id (discover ids via organizations mode)
• license — license id (e.g. lov2, cc-by, odc-odbl)

Example input

{
  "portalUrl": "https://www.data.gouv.fr",
  "mode": "datasets",
  "query": "transport",
  "maxRecords": 500,
  "pageSize": 50
}

How it works

• Server-driven pagination. uData paginates with a 1-indexed page + page_size; the actor increments page and stops the instant the server returns next_page: null (or an empty page) — never an offset guess, never an over-read. The server's total is logged as the authoritative count.
• Nested resources are flattened. Each dataset's resources[] (the actual files) are summarised onto the dataset row, with the full array kept losslessly under resources.
• Everything optional is nullable. Temporal / spatial / frequency / license fields are frequently absent and the field set differs between portals, so all three record types share one flat, fully-nullable superset schema and every optional field is a [type, null] union.
• Honest results. A reachable portal matching zero records returns zero rows (exit 0) — never a fabricated row. A portalUrl whose /api/1/ surface can't be reached at all fails loud (exit 91); an unreachable portal is never reported as an empty result.

Output

One flat record per item. Common fields: recordType, id, slug, title, description, orgName, tags, license, frequency, resourceCount, firstResourceUrl, firstResourceFormat, reuseType, topic, metricViews, plus _portalUrl / _mode / _apiUrl / _rowIndex provenance and the lossless _raw source object.

Pricing

Pay-per-event: $0.0005 per record (dataset / organization / reuse row, plus resource rows when emitResources is on), with volume discounts across subscription tiers.

Your maximum charge is respected

Each record is billed in the same step that writes it, so you never get unpaid rows or rows you were not billed for. Before every page request the run checks that at least one more record fits your Max total charge and maxRecords; when it doesn't, the run stops and makes no further request to the portal. maxRecords is a strict limit that also counts resource rows. A billing error stops the run with a failure instead of reporting success, and a restarted run never writes or bills a record twice.

Notes

• These are government servers — keep a descriptive userAgent. No auth is required for public reads.
• transport.data.gouv.fr uses a different specialised /api/datasets shape and is out of scope; this actor targets the standard /api/1/ uData core.