# Eu Open Data Portal Scraper (`bigibyte/eu-open-data-portal-scraper`) Actor EU Open Data Portal Scraper searches the public data.europa.eu hub and saves dataset records you can export to JSON, CSV, or Excel. Search by keyword, filter by country or format, or look up a dataset id. No login needed. It paces requests, retries rate limits, and skips duplicate dataset ids. - **URL**: https://apify.com/bigibyte/eu-open-data-portal-scraper.md - **Developed by:** [BigiByte](https://apify.com/bigibyte) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## EU Open Data Portal Scraper: Search data.europa.eu datasets

EU Open Data Portal Scraper

EU Open Data Portal Scraper searches the public data.europa.eu hub and saves dataset records as a flat dataset. You can search by keyword, narrow by country or file format, or look up a dataset id. No login. Results export to JSON, CSV, or Excel from the Apify dataset. ### Use cases - **Open data research**: pull air quality, transport, or other data.europa.eu datasets into a spreadsheet - **Catalog monitoring**: schedule a keyword search and compare new dataset ids over time - **Country filters**: keep German, Belgian, or other national catalogs by ISO code - **Format checks**: ask for CSV or JSON distributions when you only want files you can load - **Id lookup**: confirm a known dataset id is still present on the hub - **Developer pipelines**: feed the dataset into Make, Zapier, or a warehouse without writing a client ### Input | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `searchQueries` | array | air quality | Keywords to search for. One per line. | | `countries` | array | empty | Lower-case ISO2 country codes, such as `de`. | | `formats` | array | empty | File format labels, such as `CSV`. | | `datasetIds` | array | empty | Dataset ids to look up. Unknown ids become error rows. | | `maxItems` | integer | 100 | Cap on saved dataset records. Maximum 1000. Error rows are extra. | | `requestDelaySecs` | number | 0.3 | Pause between requests, in seconds. Minimum 0.3. | | `timeoutSecs` | integer | 300 | Stop starting new requests after this many seconds. | | `requestTimeoutSecs` | integer | 30 | Timeout for each request. | | `proxyConfiguration` | object | `{"useApifyProxy": false}` | Optional proxy. Default is no proxy. | #### Example input ```json { "searchQueries": ["air quality"], "countries": [], "formats": [], "datasetIds": [], "maxItems": 100, "requestDelaySecs": 0.3, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } } ``` A country facet run looks like this: ```json { "searchQueries": ["transport"], "countries": ["de"], "formats": ["CSV"], "maxItems": 20 } ``` ### What data does this actor extract? Each saved dataset record keeps the hub fields, plus `source_url`, `scraped_at`, and `mode`. The hub names `title` and `description` are stored as `itemTitle` and `itemDescription`. `mode` is `dataset search`, `dataset detail`, or `facet filter`. ```json { "id": "be-air-example", "itemTitle": { "en": "Air quality" }, "itemDescription": { "en": "Belgian open data record" }, "country": { "label": "Belgium", "id": "be" }, "catalog": { "id": "data-gov-be" }, "keywords": [{ "id": "air", "label": "air", "language": "fr" }], "modified": "2024-01-01T00:00:00Z", "resource": "http://data.europa.eu/88u/dataset/be-air-example", "source_url": "https://data.europa.eu/data/datasets/be-air-example", "scraped_at": "2026-10-12T00:00:00+00:00", "mode": "dataset search" } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string | Dataset id, also the dedupe key. | | `itemTitle` | object | Multilingual title from the hub. | | `itemDescription` | object | Multilingual description from the hub. | | `country` | object | Country label and code. | | `catalog` | object | Catalog that harvested the dataset. | | `keywords` | array | Keyword objects. | | `distributions` | array | Files and access URLs, including their format labels. | | `resource` | string | Canonical dataset URI. | | `source_url` | string | Source URL for the row. | | `scraped_at` | string | UTC time the row was saved. | | `mode` | string | dataset search, dataset detail, or facet filter. | | `error` | string | Present only on a failed lookup. | | `input` | string | Query or id that failed. | | `status` | integer | HTTP status, or 0 when there was no response. | Failed lookups do not stop the run. An unknown dataset id is stored as `{ "error", "input", "status": 404 }`. ### Limits and terms European Commission metadata on the portal is reusable under the Commission reuse decision, and EC content is offered under CC BY 4.0. Each dataset can carry its own licence. Check the record before you republish files. No account or login is required. ### FAQ **Does this actor need an account or login?** No. The data.europa.eu portal is public, so you do not log in. **How fresh is the data?** Rows reflect what the portal shows at run time. `modified` and `issued` are whatever the portal last published. Schedule the actor if you want a newer pull tomorrow. **What happens on an empty search?** If you clear keywords, countries, formats, and dataset ids, the run finishes with no rows and a warning in the log. **Can I export CSV?** Yes. Open the run dataset on Apify and export JSON, CSV, or Excel. The overview view shows id, title, country, catalog, keywords, modified, resource, and mode. ### Related tools If you plan to join or analyse these rows, a quick CSV cleanup or a de-duplication pass are the usual next steps. BigiByte also builds scrapers for other open-data portals when you need them as a separate job. Need another national or EU open-data catalog, a scheduled monitor, or a custom pipeline? BigiByte builds custom scrapers and data tools: https://bigibyte.com #### Trademark Notice EU Open Data Portal and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of the data and does not imply affiliation, sponsorship, endorsement, or partnership. #### Disclaimer Disclaimer: This Actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or otherwise officially associated with EU Open Data Portal or its owners. All data is collected from publicly available sources. Users are responsible for complying with the source's terms of use, rate limits, and attribution or licensing requirements. #### Is web scraping legal? This Actor only collects publicly available data from EU Open Data Portal, without logging in or bypassing any access controls. Scraping public data is generally legal, but how you use it matters: - **Personal data:** If the results include personal data (for example names, emails, or profile details), laws such as the GDPR in the EU and the CCPA in California may apply. Only collect and store personal data when you have a legitimate reason. - **Copyright:** Some content (articles, images, descriptions) may be protected by copyright. Do not republish full copyrighted content without permission. - **Terms of use:** Respect the source's terms, stated rate limits, and any attribution or licence requirements listed in the Limits and terms section. - **Legal advice:** This is general information, not legal advice. If you're unsure, consult a lawyer. To learn more, read Apify's guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/) # Actor input Schema ## `searchQueries` (type: `array`): Keywords sent as the q parameter on the public data.europa.eu hub search. One query per line. Leave empty when you only want a country or format facet, or a dataset id lookup. ## `countries` (type: `array`): ISO 3166-1 alpha-2 country codes in lower case, used as the country facet. Example: de for Germany, be for Belgium. Leave empty to skip the country facet. ## `formats` (type: `array`): Distribution format labels for the format facet, such as CSV, JSON, or XML. Leave empty to skip the format facet. ## `datasetIds` (type: `array`): Dataset ids or slugs to look up on the same hub search endpoint. A record is kept only when its id, identifier, or resource matches. An unknown id produces one error row and the run continues. ## `maxItems` (type: `integer`): Maximum number of dataset records to save in one run. Error rows do not count toward this cap. Hard maximum is 1000. ## `requestDelaySecs` (type: `number`): Seconds to wait between hub search requests. Minimum is 0.3 so calls stay sequential and paced. ## `timeoutSecs` (type: `integer`): Overall run time limit in seconds. The actor stops starting new requests after this deadline and flushes records already collected. ## `requestTimeoutSecs` (type: `integer`): Per-request timeout in seconds for each call to the hub search endpoint. ## `proxyConfiguration` (type: `object`): Optional proxy for hub search requests. Default is no proxy. Turn on Apify Proxy only if you need another IP. ## Actor input object example ```json { "searchQueries": [ "air quality", "transport" ], "countries": [ "de" ], "formats": [ "CSV" ], "datasetIds": [ "9eba8469-0525-4e9d-9d88-87dc6295af33" ], "maxItems": 100, "requestDelaySecs": 0.3, "timeoutSecs": 300, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `datasetUrl` (type: `string`): Default dataset for this run, with one row per dataset record or error. ## `overviewUrl` (type: `string`): Table of id, title, country, catalog, keywords, modified, resource, and mode. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQueries": [ "air quality" ] }; // Run the Actor and wait for it to finish const run = await client.actor("bigibyte/eu-open-data-portal-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQueries": ["air quality"] } # Run the Actor and wait for it to finish run = client.actor("bigibyte/eu-open-data-portal-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQueries": [ "air quality" ] }' | apify call bigibyte/eu-open-data-portal-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,bigibyte/eu-open-data-portal-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/M8VrNRHEuRsluhhMm/builds/UkI64ANzOuf6NXYnU/openapi.json