OpenDataSoft Explore Extractor

What does OpenDataSoft Explore Extractor do?

OpenDataSoft Explore Extractor pulls datasets and row data from public OpenDataSoft open-data portals through their Explore API v2.1. Many city, energy, transport and public-sector portals run on OpenDataSoft, for example opendata.paris.fr and opendata.reseaux-energies.fr. You give it a portal URL. It can list the datasets published there, or pull the rows of one dataset with optional filters, sorting and aggregation.

The Actor only reads what a portal already serves through its public Explore API, plus key-protected data if you supply that portal's own API key. It does not get around logins or access controls. Portals that don't run OpenDataSoft, or that have switched the Explore API off, won't work.

Quick start: the exact sample run

The Store sample reads the first 25 rows of Paris's street-tree dataset (les-arbres):

{
  "datasetId": "les-arbres",
  "limit": 100,
  "maxRecords": 25,
  "mode": "records",
  "portalUrl": "https://opendata.paris.fr"
}

1. Click Try for free (or open the Input tab). The sample input above is already filled in.
2. Click Start. In our cloud check (run kg7u8dzu4KfARcghX) this returned 25 rows and charged 25 record events.
3. Open the Output tab and export as JSON, CSV, Excel, XML or HTML, or read the dataset through the Apify API.

To use your own data, change portalUrl to another OpenDataSoft portal, run catalog mode to find a datasetId, then switch to records mode.

Catalog mode vs records mode

Mode	What you get	One output row =
catalog	The datasets published on the portal: id, title, description, themes, keywords, publisher, license, last modified date, record count, field names, plus the raw metadata	one dataset
records	The actual rows of one dataset (needs datasetId)	one data row, or one group when you use groupBy
auto (default)	records when datasetId is set, otherwise catalog	—

Input

Only portalUrl is required. The fields you'll use most:

• portalUrl: the portal root, such as https://opendata.paris.fr. You can also paste a dataset page link straight from your browser, such as https://opendata.paris.fr/explore/dataset/les-arbres/table/. The Actor keeps the portal root and, when datasetId is empty, pulls that dataset's rows. The run log shows how the link was read.
• mode: auto, catalog or records.
• datasetId: the dataset to read in records mode, for example les-arbres.
• where: an ODSQL filter, for example hauteurenm > 20, or records_count > 1000 in catalog mode.
• select, groupBy, orderBy: ODSQL projection, aggregation and sorting. groupBy is meant for records mode, e.g. groupBy = arrondissement with select = count(*) as n gives one row per district.
• refine / exclude: facet filters written facet:value, for example domanialite:Alignement.
• maxRecords (default 1000) and limit (page size, at most 100).
• Optional: lang, timezone, queryParams (extra query keys), apikey (only for key-protected portals, never logged) and extraHeaders.

Field names depend on the dataset. Run catalog mode or look at the dataset on the portal to see which fields you can filter on.

Output

Records mode returns one row per data row. Each row keeps the dataset's own fields unchanged under record, and adds longitude/latitude when it finds a point location. This excerpt is the first row of the sample run above:

{
  "_portal": "https://opendata.paris.fr",
  "_mode": "record",
  "datasetId": "les-arbres",
  "recordId": "2c338f88b5aedd6ab2b5d9dd3b450e257973b7c9",
  "odsRecordId": "2c338f88b5aedd6ab2b5d9dd3b450e257973b7c9",
  "recordIndex": 0,
  "longitude": 2.413080589624021,
  "latitude": 48.850796202527974,
  "fieldCount": 16,
  "record": {
    "idbase": 189361,
    "arrondissement": "PARIS 20E ARRDT",
    "adresse": "ECOLE ELEMENTAIRE / 20 RUE MARYSE HILSZ",
    "libellefrancais": "If",
    "genre": "Taxus",
    "espece": "baccata",
    "circonferenceencm": 140,
    "hauteurenm": 6,
    "stadedeveloppement": "Adulte",
    "geo_point_2d": { "lon": 2.413080589624021, "lat": 48.850796202527974 }
  }
}

(The real row also has fieldKeys and a few more tree fields; we shortened it here.) Fields inside record are different for every dataset, and longitude and latitude are null when the Actor can't find them. odsRecordId is the portal's own id for the row (the same id its record API uses), so you can match rows between runs; recordId is the dataset's own id column when there is one, otherwise that portal id. Both are null for groupBy results, which have no single row. Catalog rows use the columns listed in the table above.

Paging and limits

• The Actor pages through results for you, up to 100 rows per request, until it reaches maxRecords, runs out of rows, or runs out of your budget.
• OpenDataSoft only lets you page through the first 10,000 rows of a query (offset + limit ≤ 10000). To get more, split the work into several runs with where filters (by date, district, and so on). orderBy keeps each slice in a fixed order, so you can also run ascending and descending to reach both ends.
• If a filter matches nothing, the run finishes cleanly with 0 rows and costs nothing for records.
• Portal rate limits and outages are outside our control. The Actor makes up to 4 attempts on rate-limit, server and network errors. Errors such as a bad filter or a missing dataset stop the run straight away.

Pricing and your maximum charge

This OpenDataSoft extractor charges one record event per row it delivers (US$0.0005 per record on the FREE tier, lower on paid tiers; platform usage is billed separately). If you set Max total charge on a run, the Actor asks the portal only for as many rows as that budget and Max records still cover, stops before the next page once the budget is used, and makes no request at all if the budget cannot pay for one row. Every delivered row is charged; if a portal page fails after some rows were delivered, you keep (and pay for) those rows and the run ends with an error.

Price per record by Apify plan: FREE US$0.0005, BRONZE US$0.00045, SILVER US$0.0004, GOLD US$0.00035, PLATINUM US$0.0003, DIAMOND US$0.00025. On the FREE tier, the 25-row sample costs US$0.0125 in record events and 1,000 rows cost US$0.50, plus platform usage.

Filtered CSV without code (Console)

This gets the same 10 remarkable trees of Paris as the Python recipe below, with no code.

Step 1. Click Try for free. On the Input tab, switch from Form to JSON and replace the text with this (the copy button in the box's top-right corner copies all of it, even if a line looks cut off on a phone):

{
  "portalUrl": "https://opendata.paris.fr",
  "mode": "records",
  "datasetId": "les-arbres",
  "where": "remarquable = 'OUI'",
  "orderBy": "idbase",
  "maxRecords": 10
}

In the Form view, the same values go in OpenDataSoft portal URL, Mode, Dataset id (records mode), ODSQL where filter, ODSQL order_by (sort) and Max records (total).

Step 2 (optional money cap). Under Run options, set Maximum cost per run (for example $0.01). That is an Apify run setting, not an Actor input, so don't add it to the JSON.

Step 3. Click Start (the owner's Console shows Save & start). Our Console run 4mVRd6ZIumsytXU44 returned 10 rows and charged 10 record events.

Step 4. On the Output tab, click Export, change View from OpenDataSoft records to All fields (the default view has no odsRecordId column), pick CSV, then Download.

What's in the file:

• One row per tree, the first 10 matches by idbase.
• Tree fields as record/<field> columns (record/idbase, record/libellefrancais, record/hauteurenm …), location as record/geo_point_2d/lat and /lon.
• odsRecordId: the portal's own row id. Empty values stay empty cells.
• fieldKeys/0, fieldKeys/1 … list the field names. Drop them with Omit fields → fieldKeys.

remarquable = 'OUI' only works on this Paris dataset. For another dataset, check its field names on the portal (or run catalog mode) and write your own where and orderBy.

Export filtered rows to CSV with Python (API recipe)

This script uses only the Python standard library. It starts one run for the remarkable trees of Paris (remarquable = 'OUI' in the public les-arbres dataset), sorted by idbase. It waits up to 5 minutes, then writes the rows to paris_remarkable_trees.csv. Set APIFY_TOKEN in your environment first. The script sends it as a Bearer header and never prints it.

• maxRecords: 10 caps the rows. maxTotalChargeUsd=0.01 separately caps what this run can charge in record events. Platform usage is billed on top as usual. The where filter is specific to this Paris dataset.
• Columns are _portal, datasetId, odsRecordId, recordId, then every field the portal actually returned. Missing values stay as empty cells and nothing is filled in or guessed. Nested values such as geo_point_2d are written as JSON text.
• The CSV is replaced only after a SUCCEEDED run with valid rows. On an API error, a failed or aborted run, or a run still going after 5 minutes, the script exits non-zero and leaves any existing CSV as it was. It does not start another run by itself, so check the run in Console first. A successful run with no matches writes a header-only file.
• The result is a capped snapshot of the first matching rows at run time. It is not a full census of the dataset and not a change feed. Change where, orderBy and maxRecords to fit your own portal and dataset.

import csv, http.client, json, os, sys, time, urllib.request

TOKEN = os.environ["APIFY_TOKEN"]  # read from the environment, never printed
API = "https://api.apify.com/v2"
RUN_INPUT = {
    "portalUrl": "https://opendata.paris.fr",
    "mode": "records",
    "datasetId": "les-arbres",
    "where": "remarquable = 'OUI'",  # only trees Paris flags as remarkable
    "orderBy": "idbase",             # stable order, so the same 10 come back first
    "maxRecords": 10,                # row cap
}
MAX_CHARGE_USD = 0.01  # separate money cap on this run's Actor event charges
OUT = "paris_remarkable_trees.csv"

def call(method, path, body=None):
    req = urllib.request.Request(API + path, method=method,
        data=None if body is None else json.dumps(body).encode(),
        headers={"Authorization": "Bearer " + TOKEN, "Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.load(r)

try:
    actor_id = call("GET", "/acts/datamule~opendatasoft-explore-extractor")["data"]["id"]
    run = call("POST", f"/acts/{actor_id}/runs?maxTotalChargeUsd={MAX_CHARGE_USD}", RUN_INPUT)["data"]
    deadline = time.time() + 300
    while run["status"] in ("READY", "RUNNING") and time.time() < deadline:
        time.sleep(5)
        run = call("GET", f"/actor-runs/{run['id']}")["data"]
    if run["status"] in ("READY", "RUNNING"):
        sys.exit(f"Run {run['id']} is still {run['status']} after 5 minutes and may still finish "
                 f"(within the charge cap); {OUT} was not changed. Check it in Console before starting another.")
    if run["status"] != "SUCCEEDED":
        sys.exit(f"Run {run['id']} ended {run['status']}; {OUT} was not changed.")
    rows = call("GET", f"/datasets/{run['defaultDatasetId']}/items?clean=true&format=json")
except (OSError, http.client.HTTPException, KeyError, ValueError) as e:
    sys.exit(f"API error ({type(e).__name__} {getattr(e, 'code', '')}); {OUT} was not changed.")

if not isinstance(rows, list) or any(
        not isinstance(r, dict) or r.get("_mode") != "record" or not isinstance(r.get("record"), dict)
        for r in rows):
    sys.exit(f"Unexpected rows in run {run['id']}; {OUT} was not changed.")

meta = ["_portal", "datasetId", "odsRecordId", "recordId"]
cols = []
for r in rows:  # every field the portal actually returned, in first-seen order
    cols += [k for k in r["record"] if k not in cols]

def cell(v):  # None -> empty cell; nested values (e.g. geo_point_2d) -> JSON text
    return "" if v is None else json.dumps(v, ensure_ascii=False) if isinstance(v, (dict, list)) else v

try:
    with open(OUT + ".tmp", "w", newline="", encoding="utf-8") as f:
        w = csv.writer(f)
        w.writerow(meta + cols)
        for r in rows:
            w.writerow([cell(r.get(k)) for k in meta] + [cell(r["record"].get(k)) for k in cols])
    os.replace(OUT + ".tmp", OUT)  # the old file is replaced only after a valid SUCCEEDED run
finally:
    if os.path.exists(OUT + ".tmp"):
        os.remove(OUT + ".tmp")
print(f"Run {run['id']}: {len(rows)} rows -> {OUT}")

Exporting and recurring use

• Download any run's dataset as JSON, CSV, Excel, XML or HTML, or fetch it through the Apify API.
• To refresh data regularly, save your input as a Task and add an Apify schedule yourself. Each scheduled run is billed like a manual one, so set maxRecords and Max total charge first.
• Integrations (webhooks, Make, Zapier, Google Drive and others) can pick up each finished dataset.

FAQ and support

Does it work on every OpenDataSoft portal? Only on portals that expose the Explore API v2.1 publicly, or to your API key. Some portals switch it off or restrict datasets.

Who owns the data? Each portal publishes under its own license (catalog rows include a license field). Check that license before you reuse the data.

Something broke? Open an issue on the Issues tab and include the portal URL and your input (leave out any API key).