uData Portal Extractor

Point at any uData open-data portal and get clean, structured rows. uData (built by Etalab) is the platform behind data.gouv.fr and roughly 30 francophone / EU government open-data portals, and every deployment exposes the same uniform /api/1/… REST API — so this is one generic runner, not a per-portal scraper. Give it a portal base URL and a mode.

uData is distinct from CKAN, Socrata, and OpenDataSoft. Its /api/1/ shape, its 1-indexed page / page_size pagination, its nested per-dataset resources[], and its community reuses graph are uData-specific.

Modes

mode	Endpoint	One row per…
datasets (default)	/api/1/datasets/	dataset — with its resources[] summarised on (resourceCount, first-resource url/format/filesize/mime) and the full lossless resources array kept
organizations	/api/1/organizations/	publishing organisation (name, website, dataset/reuse/member counts)
reuses	/api/1/reuses/	community reuse — an app / article / visualisation built on the portal's data (a uData-unique layer)

With emitResources on (datasets mode), one extra row per resource — each an actual downloadable file (url / format / filesize / checksum / mime) — is also emitted, joined to its parent dataset.

Facet filters

All passed straight through to the uData API and combinable:

• query — full-text q= filter
• tag — tag facet
• organization — an organization id (discover ids via organizations mode)
• license — license id (e.g. lov2, cc-by, odc-odbl)

Example input

{
  "portalUrl": "https://www.data.gouv.fr",
  "mode": "datasets",
  "query": "transport",
  "maxRecords": 500,
  "pageSize": 50
}

How it works

• Server-driven pagination. uData paginates with a 1-indexed page + page_size; the actor increments page and stops the instant the server returns next_page: null (or an empty page) — never an offset guess, never an over-read. The server's total is logged as the authoritative count.
• Nested resources are flattened. Each dataset's resources[] (the actual files) are summarised onto the dataset row, with the full array kept losslessly under resources.
• Everything optional is nullable. Temporal / spatial / frequency / license fields are frequently absent and the field set differs between portals, so all three record types share one flat, fully-nullable superset schema and every optional field is a [type, null] union.
• Honest results. A reachable portal matching zero records returns zero rows (exit 0) — never a fabricated row. A portalUrl whose /api/1/ surface can't be reached at all fails loud (exit 91); an unreachable portal is never reported as an empty result.

Output

One flat record per item. Common fields: recordType, id, slug, title, description, orgName, tags, license, frequency, resourceCount, firstResourceUrl, firstResourceFormat, reuseType, topic, metricViews, plus _portalUrl / _mode / _apiUrl / _rowIndex provenance and the lossless _raw source object.

Pricing

Pay-per-event: $0.0005 per record (dataset / organization / reuse row, plus resource rows when emitResources is on), with volume discounts across subscription tiers.

Notes

• These are government servers — keep a descriptive userAgent. No auth is required for public reads.
• transport.data.gouv.fr uses a different specialised /api/datasets shape and is out of scope; this actor targets the standard /api/1/ uData core.