# GovData Germany Scraper - CKAN Open Data Portal (`parseforge/govdata-germany-scraper`) Actor

Scrape Germany's national open data portal: 156,000+ Datensätze with full DCAT-AP.de metadata, distributions, publishers, EU data themes, High-Value Dataset flags, licences and live link checks.

- **URL**: https://apify.com/parseforge/govdata-germany-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 50.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### GovData Germany Scraper - CKAN Open Data Portal

**Scrape GovData.de, Germany's national open data portal.** 156,461 Datensätze with the full DCAT-AP.de record — publisher, contact point, EU data theme, keywords, licence, temporal and spatial coverage, update frequency, High-Value Dataset flag — plus every distribution, and optional checks that tell you whether the file behind the link is actually there. Eight datasets behind one input. No login, no API key. Export to CSV, JSON, Excel or XML.

GovData is the single index of German public-sector open data: federal ministries, all sixteen Länder, the big city portals and the geodata infrastructure all harvest into it. The portal has a CKAN API, but it hands you raw CKAN — the DCAT-AP.de payload arrives as a flat key/value list with JSON arrays serialised into strings, the format field holds 403 different spellings of about thirty formats, and nothing tells you whether the download still works. This Actor turns that into typed rows.

| Who uses it | What they scrape GovData for |
|---|---|
| Data engineers and analysts | A queryable index of every German public dataset, filtered to the formats they can actually ingest |
| GovTech and civic-tech teams | Monitoring what each Land and Kommune publishes, and when it changes |
| Researchers and journalists | Finding the Statistik or Geodaten behind a story, with the publisher's contact details attached |
| Compliance and policy teams | The 10,514 EU High-Value Datasets and their legal basis, by category |
| Data catalogue vendors | DCAT-AP.de metadata to seed or reconcile their own catalogue |
| Lead generation for public sector | 175 distinct publishing bodies per 1,400 datasets, with email, phone and postal address |

### What it does

Pick a **dataset**, set filters, get rows. Every row carries the GovData page URL and a scrape timestamp.

- 🗂️ **Datasets** — one row per Datensatz, **84 fields**: title, description, publisher, contact point with postal address, EU data theme, keywords, licences, issued and modified dates, temporal coverage, bounding box and centroid, update frequency, administrative level, regional keys (AGS), High-Value Dataset flag and legal basis, source portal, and every distribution nested with its own metadata.
- 📦 **Distributions** — one row per file or service, **42 fields**: URL, normalised format, MIME type, byte size, checksum, licence and attribution text, dates — carrying the identity, publisher, themes and keywords of the dataset it belongs to. Datasets average 4.0 distributions each.
- 📇 **Contact points** — the publishing bodies deduplicated out of whatever you scanned: name, email, phone, street, postcode, city, how many datasets they are responsible for and which themes those cover.
- 🏛️ **Organizations** — all **76** publishing organizations with dataset counts, both catalogue-wide and within your filters.
- 🏷️ **Categories** — the **13** EU data themes with German and English titles, their data-theme URIs and counts.
- 🔖 **Tags** — the keyword vocabulary (**97,863** tags) ranked by dataset count, honouring your filters.
- 📄 **Formats** — the distribution format directory, grouped onto a normalised name with every raw spelling the catalogue uses underneath it.
- ⚖️ **Licences** — the **52** registered licences with Open Definition and Open Source conformance.

### What is in the catalogue

Measured 2026-08-27 against the live index:

| | Datasets |
|---|---|
| Total | **156,461** |
| With at least one CSV distribution | 68,582 |
| With a geometry (bounding box + centroid on the row) | 72,621 |
| With a temporal coverage window | 68,163 |
| With a machine-readable contact email | 93,876 |
| EU High-Value Datasets | 10,514 |
| A WMS, WFS, WCS or WMTS service | 14,559 |
| JSON or GeoJSON | 13,675 |
| Metadata changed in the last 7 days | 50,773 |
| First published in the last 30 days | 6,894 |

Largest publishers: GDI-DE (27,596), Land Schleswig-Holstein (27,546), open.bydata / Bayern (26,634), Open Data Brandenburg (19,445), Transparenzportal Hamburg (13,965), Open.NRW (10,407), Land Rheinland-Pfalz (9,221).

### What you can do with GovData data

**🔎 Find the datasets nobody can find on the portal.**

`ckan` is a 6,600-a-month search and `open data portal` 1,000 at a $14.37 CPC — people are looking for exactly this index. GovData's own search is one box over 156,461 records. Here you can ask for *datasets in the Umwelt theme, published by a Land, updated in the last month, that ship a CSV, with a contact email* and get 84 typed columns back.

**📊 Build a column index of German open data.**

`datensatz` runs 1,000 searches a month in Germany at a $13.40 CPC, and the reason is that finding the right one is hard. Switch on **Preview distribution columns** and every CSV, TSV, JSON, GeoJSON or XML distribution comes back with its real column names, delimiter, character encoding and a sample row — read from the first 64 KB with a range request, so a 600 MB file costs 64 KB. Now "which datasets have a `gemeindeschluessel` column" is a query, not an afternoon.

**🔗 Find the dead links before your pipeline does.**

GovData harvests metadata from regional portals and never re-verifies it. Measured across 150 distributions spread over the full age range of the catalogue, **5.3% were dead** — plain HTTP 404s, including federal budget files, with nothing in the metadata to warn you. **Check that distribution links are alive** attaches status, content type, byte size, redirect target and latency to every URL, and **Drop rows whose links are dead** removes the corpses.

**⚖️ Pull the EU High-Value Datasets and their legal basis.**

10,514 datasets carry the HVD flag from Implementing Regulation (EU) 2023/138 — the categories member states must publish openly and free of charge. Filter by category (earth-environment 9,419 · geospatial 603 · mobility 313 · statistics 217 · meteorological 81 · companies 5) and each row carries the `applicable_legislation` URI it was published under.

**🗺️ Get geodata coverage without a GIS library.**

72,621 datasets carry a spatial extent. Rows give you `boundingBox`, `centroidLon` and `centroidLat` computed from the geometry, plus the raw GeoJSON, the AGS regional keys and the administrative level (Bund / Land / Kreis / Gemeinde). `geodaten deutschland` and `inspire geodaten` are 110 and 40 searches a month in Germany.

**📬 Build a public-sector contact list.**

The **Contact points** dataset deduplicates publishers, contacts, authors and maintainers out of whatever you scanned. A 1,400-dataset sample held 175 distinct publishing bodies. Each row carries email, phone, street, postcode and city where the publisher supplied them — 93,876 datasets carry a machine-readable contact email.

**🔔 Run it as a change monitor.**

Set **Metadata modified after** to yesterday and sort by recently updated: 50,773 records changed in the last 7 days. Set **First published after** instead and you get genuinely new datasets — 6,894 in the last 30 days — rather than re-harvested ones.

### Why choose this scraper

| | What you get |
|---|---|
| **Eight datasets, one Actor** | Datasets, distributions, contacts, organizations, categories, tags, formats and licences. |
| **84 fields, not raw CKAN** | The DCAT-AP.de payload arrives as a flat key/value list with JSON arrays serialised into strings. Every one is parsed out into a named, typed field. |
| **Filters that work** | Every filter here was measured against the unfiltered total. Three of the obvious ones are broken upstream and are handled or excluded — see below. |
| **Nothing always-null shipped** | Field coverage was measured across a 2,000-dataset sample spread over the whole catalogue before any column was included. |
| **Link checking and column previews** | Both optional, both off by default, both billed only for work actually done. |
| **Filters run before the extras are bought** | Narrow by licence, format or text on the complete row; a row you filter out never costs a link check. |
| **No login, no API key, no proxy** | 164 requests at up to 16 concurrent workers ran clean from a bare datacenter IP with no throttling. |
| **Ends cleanly at your budget** | At Apify's per-run charge ceiling the run stops and finishes SUCCEEDED with what it has, instead of being aborted. |

#### Four traps this Actor handles for you

**The format filter is a lie if you take it literally.** `res_format` holds the raw string each harvesting portal supplied, with no normalisation — 403 distinct values for about thirty real formats, 103 of them full EU file-type URIs and 300 free text. Filtering `res_format:"CSV"` looks right and returns 7,859 datasets. The real number with a CSV distribution is **68,582**: most carry `http://publications.europa.eu/resource/authority/file-type/CSV` instead, plus lowercase `csv`, `.csv` and `text/csv`. Every format name you type is expanded to all of them, and the **Formats** dataset shows you each variant with its own count.

**Spatial bounding-box search silently returns everything.** CKAN's `ext_bbox` parameter is accepted on this portal and answers the full 156,461 for any box you give it — the spatial extension is not installed. A bbox filter would look like it worked and quietly not filter. There isn't one here. Use **Regional keys**, **Administrative level** or the `boundingBox` on the row instead.

**An unknown sort field does not error, it reverts to relevance.** `views_recent desc` and `relevance asc` are both accepted and both return the identical first row as `score desc`. Only the eight sort orders verified to actually change the order are offered.

**The licence is not on the dataset.** Package-level `license_id` is populated on 1,245 of 156,461 records — 0.8%. The licence that governs the data sits on each distribution, where it is populated everywhere. Rows carry it as `licenses` / `licenseCodes` on the dataset and `license` on each distribution, and the **Licences** filter matches there.

### How it compares

**No other Apify Actor scrapes GovData.de.** We searched the Store for `govdata`, `open data germany`, `german open data`, `ckan` and `dcat`. What exists is generic CKAN and Socrata exporters that point at any portal and hand back the raw package.

| Feature | ParseForge (this) | doggo/ckan-opendata-exporter | straightforward\_hydra/ckan-open-data-scraper | datapilot/open-data-portal-harvester | crawlerbros/eu-open-data-scraper |
|---|---|---|---|---|---|
| Source | GovData.de | Any CKAN | Any CKAN | Any portal | EU Open Data Portal |
| DCAT-AP.de fields parsed out | 84 | Raw CKAN | Raw CKAN | Raw | Raw |
| Format filter expanded to all 403 spellings | Yes | No | No | No | No |
| High-Value Dataset flag and category | Yes | No | No | No | No |
| Bounding box + centroid computed | Yes | No | No | No | No |
| Regional keys (AGS) and administrative level | Yes | No | No | No | No |
| Distribution link checking | Optional | No | No | No | No |
| Column preview of the actual file | Optional | No | No | No | No |
| Contact-point directory | Yes | No | No | No | No |
| Organization / theme / tag / format / licence directories | 5 datasets | No | No | No | No |
| Price per 1,000 rows | $7.00 | $1.00 | $2.00 | $2.00 | $5.00 |

**Why $7 per 1,000 when generic CKAN exporters charge $1.** They are not selling the same row. Point a generic exporter at `www.govdata.de/ckan` and you get CKAN's `package_search` output: `extras` as a key/value list, `["http://publications.europa.eu/resource/authority/data-theme/ENVI"]` as a string inside a string, no HVD flag, no bounding box, no AGS, no idea which of the 403 format spellings to filter on, and no signal that 5.3% of the links are dead. Getting from that to something you can query is the work this Actor did once. If you want a raw CKAN dump of any portal, buy the $1 exporter — it is the right tool for that job.

This Actor is HTTP-only with no browser and no proxy. Measured on the platform:

| Run | Rows | Our cost per 1,000 rows |
|---|---|---|
| Datasets, 5,000-row sweep | 5,000 | $0.0054 |
| Datasets, default input | 100 | $0.0080 |
| Distributions with link check **and** column preview | 120 | $0.0171 |
| Organizations with profiles (worst case: fixed cost over 6 rows) | 6 | $0.0496 |

Even the deliberately worst-shaped run — six rows, every optional block on — costs us five cents per thousand. We are not pricing on cost; the house floor is $5 and this row is not a commodity.

### What a row looks like

**Datasets** (`dataset: datasets`) — 84 fields, abridged:

```json
{
  "id": "182ec075-3a65-4756-9ee5-059cc888d1ae",
  "name": "mobilpunkte",
  "title": "Mobilpunkte",
  "notes": "Standorte der Mobilpunkte in der Hanse- und Universitätsstadt Rostock",
  "url": "https://www.govdata.de/suche/daten/mobilpunkte",
  "landingPage": "https://www.opendata-hro.de/dataset/9c32aba3-4380-4372-b31c-d0ba53fba326",
  "organization": "opendata-hro",
  "organizationTitle": "OpenData.HRO",
  "publisherName": "Hanse- und Universitätsstadt Rostock",
  "sourcePortal": "http://www.opendata-hro.de/",
  "categories": ["gove", "regi", "envi", "tran"],
  "categoryLabels": ["Regierung und öffentlicher Sektor", "Regionen und Städte", "Umwelt", "Verkehr"],
  "themeCodes": ["GOVE", "REGI", "TRAN", "ENVI"],
  "tags": ["automobil", "bahnhof", "busbahnhof", "elektrizität", "individualverkehr"],
  "isHighValueDataset": true,
  "hvdCategories": ["http://data.europa.eu/bna/c_b79e35eb"],
  "hvdCategoryLabels": ["Mobility"],
  "applicableLegislation": ["http://data.europa.eu/eli/reg_impl/2023/138/oj"],
  "licenses": ["http://dcat-ap.de/def/licenses/cc-zero"],
  "licenseCodes": ["cc-zero"],
  "accessRights": "http://publications.europa.eu/resource/authority/access-right/PUBLIC",
  "issued": "2020-04-02T12:51:11.247095",
  "metadataModified": "2026-08-27T07:04:23.785420",
  "languageCodes": ["DEU"],
  "spatialText": "Rostock",
  "regionalKeys": ["130030000000"],
  "administrativeLevel": "administrativeDistrict",
  "boundingBox": { "west": 11.99143, "south": 54.03856, "east": 12.31306, "north": 54.26196 },
  "centroidLon": 12.152245,
  "centroidLat": 54.15026,
  "contactName": "Hanse- und Universitätsstadt Rostock",
  "contactEmail": "geodienste@rostock.de",
  "resourceCount": 7,
  "formats": ["CSV", "XLSX", "GEOJSON", "KML", "GML"],
  "totalSizeBytes": 28673,
  "resources": [
    {
      "name": "Mobilpunkte",
      "format": "CSV",
      "formatRaw": "http://publications.europa.eu/resource/authority/file-type/CSV",
      "url": "https://geo.sv.rostock.de/download/opendata/mobilpunkte/mobilpunkte.csv",
      "license": "http://dcat-ap.de/def/licenses/cc-zero",
      "sizeBytes": 6422,
      "linkCheck": {
        "status": 200, "ok": true, "contentType": "text/x-comma-separated-values",
        "contentLength": 6422, "redirected": false, "method": "HEAD", "elapsedMs": 1176
      },
      "columnPreview": {
        "ok": true, "delimiter": ",", "encoding": "utf-8", "columnCount": 12,
        "columns": ["uuid", "bezeichnung", "latitude", "longitude", "betreiber"]
      }
    }
  ],
  "scrapedAt": "2026-08-27T14:22:32.420Z"
}
```

**Formats** (`dataset: formats`) — the trap, made visible. `datasetCountSum` adds up the per-variant counts, so it reads slightly differently from the 68,582 above: that figure comes from the expanded filter, which also catches long-tail spellings like `.csv` and `gezippte csv-dateien` that fall outside the top variants. Use `largestVariantCount` when you want an exact number, and the filter when you want the rows.

```json
{
  "format": "CSV",
  "datasetCountSum": 68444,
  "largestVariantCount": 60287,
  "rank": 1,
  "variantCount": 5,
  "variants": [
    { "value": "http://publications.europa.eu/resource/authority/file-type/CSV", "count": 60287 },
    { "value": "CSV", "count": 7859 },
    { "value": "https://www.iana.org/assignments/media-types/text/csv", "count": 178 },
    { "value": "https://data-lra-cha.opendata.arcgis.com/api/feed/dcat-ap/ftype/CSV", "count": 105 },
    { "value": "http://publications.europa.eu/resource/authority/file-type/csv", "count": 15 }
  ],
  "isMachineReadable": true,
  "isService": false
}
```

**Contact points** (`dataset: contacts`):

```json
{
  "role": "contact",
  "name": "Berlin.de",
  "email": "opendata@senfin.berlin.de",
  "organization": "berlin-open-data",
  "organizationTitle": "Berlin Open Data",
  "publisherName": "Senatsverwaltung für Finanzen Berlin",
  "sourcePortal": "http://datenregister.berlin.de",
  "datasetCount": 6,
  "categories": ["gove"],
  "exampleDatasetTitle": "Aktenplan der Senatsverwaltung für Finanzen"
}
```

### Configure the run

| Input | What it does |
|---|---|
| **Dataset** | Which of the eight surfaces to pull. Each bills its own per-row event. |
| **Search query** | Full text across titles, descriptions, tags and publishers, exactly as the portal's own box works. German stemming applies — `Wasser` also matches `Gewässer`. |
| **Title contains / Description contains** | Field-scoped search, ANDed with the above. |
| **Organizations** | Publisher slugs. Run the Organizations dataset once for the list of 76 with counts. |
| **Categories** | The 13 EU data themes. |
| **Tags** | Exact keywords, matched case-sensitively. Run the Tags dataset to see what exists. |
| **Distribution formats** | `CSV`, `GEOJSON`, `WFS`, `PARQUET` … expanded to every spelling the catalogue uses. |
| **Exclude formats** | Drop datasets publishing a format you do not want. |
| **High-Value Datasets only / HVD categories** | The EU 2023/138 mandate, whole or by category. |
| **Administrative level** | Bund, Land, Kreis, Gemeinde. |
| **Regional keys (AGS)** | `130030000000` for Rostock, `11` for Berlin as a Land. |
| **Metadata language** | German covers 93,655 records; English 289. |
| **Update frequency** | Ten options, each matching both vocabularies publishers use. |
| **Publisher name / Source portal contains** | Finer than the organization: 175 distinct publishers per 1,400 datasets. |
| **Dataset type** | dct:type — dataset, series, collection. |
| **Only datasets with a geometry** | 72,621 qualify, and every row then has a bounding box. |
| **Minimum distributions** | Set 1 to exclude the 239 metadata-only records. |
| **Modified / published after and before** | ISO dates. This is the change-monitor input. |
| **Data covers time from / to** | The period the DATA covers, not when the record was edited. |
| **Licences** | Substring match on the distribution licence — `dl-by-de`, `cc-by`, `cc-zero`. |
| **Row must contain** | Plain substring over the finished row, applied before any optional block is bought. |
| **Filter query (advanced)** | Raw Solr `fq`. An unknown field name returns zero rows rather than an error. |
| **Three optional-block checkboxes** | Link check, column preview, organization profile. All off by default. |
| **Sort by** | Eight verified orders. Use *Stable id order* for full sweeps. |
| **Max items / Max catalogue pages** | Hard ceilings on rows and cost. One page is 500 datasets. |
| **Concurrency** | 5 by default, up to 12. See the throughput note below. |
| **Proxy** | Off by default. Not needed. |

#### About throughput and throttling

Measured 2026-08-27 from a bare datacenter IP, 164 requests across 1, 2, 4, 8 and 16 concurrent workers: **zero failures, zero throttles, no rate limiting of any kind.** Server latency is a flat ~2.8 s per request regardless of concurrency, so parallelism scales cleanly:

| Page size | Concurrency | Throughput |
|---|---|---|
| 500 | 8 | **525 dataset rows / s** |
| 1000 | 8 | 521 rows / s |
| 500 | 4 | 274 rows / s |
| 100 | 16 | 182 rows / s |

Above 8 workers the per-request overhead starts to dominate, which is why the input caps at 12. The portal does occasionally answer HTTP 200 with a zero-byte body under load; that is a load-shedding hiccup, not an empty page, and it is retried rather than treated as a result.

One quirk worth knowing if you script against the API yourself: **GovData answers 403 to curl's default User-Agent** on URLs that work perfectly with a browser one. This Actor always sends a browser UA.

### Pricing

Pay-per-event. You are billed per row written, plus one small charge per catalogue page fetched and a $0.054 run-start fee. Filtered-out rows cost nothing.

| Event | Price | When it fires |
|---|---|---|
| Run start | $0.054 | Once per run |
| Catalogue page scanned | $0.004 | Each page of up to 500 datasets, or one directory call |
| Dataset | $0.007 | `datasets` |
| Distribution | $0.005 | `resources` |
| Contact point | $0.006 | `contacts` |
| Organization | $0.010 | `organizations` |
| Category | $0.010 | `categories` |
| Tag | $0.003 | `tags` |
| Format | $0.003 | `formats` |
| Licence | $0.005 | `licenses` |
| Distribution link checked | $0.006 | Checkbox on, per URL checked |
| Distribution columns previewed | $0.015 | Checkbox on, per file actually read |
| Organization profiled | $0.012 | Checkbox on, per organization |

Exactly one per-row event fires on any given run — the one for the dataset you picked.

| Datasets collected | Approximate cost |
|---|---|
| 100 | $0.76 |
| 1,000 | $7.07 |
| 10,000 | $70.13 |

The three optional blocks are off by default. A run that leaves them off pays only the run-start fee, one catalogue-page event per page, and the per-row rate — verified on the platform: a default run charged `apify-actor-start=1, search-page=1, dataset-row=100` and nothing else.

**Column previews are not billed when they are skipped.** A distribution in a format that cannot be read as columns — XLSX, SHP, KML, GML, ZIP — returns early without making a request and costs nothing. Measured on the platform: 120 distributions, 120 link checks, **69** previews billed.

### Free users

Apify's free plan caps what any paid Actor can bill in a month. If a run stops early with a usage message, that is the plan cap, not this Actor. Free-plan runs here return up to 50 rows as a preview, and the log says so.

Separately, Apify gives every run a maximum cost derived from your remaining allowance. A whole-catalogue sweep is large enough to hit that on any plan. When it does, this Actor **stops and finishes cleanly with what it collected** rather than letting the platform abort the run — measured 2026-08-27: an unbounded sweep hit the ceiling at 2,200 rows and ended SUCCEEDED, where the previous build ran on to 6,100 rows and was aborted by the platform. Set **Max items** to keep a run inside a budget you choose.

### Run it

1. Pick a **Dataset** — start with *Datasets*.
2. Narrow it: a **Search query**, a **Category**, an **Organization**, or a **Distribution format**.
3. Set **Max items** if you want a hard ceiling on cost.
4. Run, then export as CSV, JSON, Excel or XML — or pull it from the API.

The default input — Umwelt datasets that publish a CSV — returns 100 rows in about four seconds from 15,935 matches.

### Troubleshooting

**Zero rows and I set a Filter query.** An unknown Solr field name returns zero rows rather than an error on this API. Check the spelling. The log says this explicitly whenever a run writes nothing.

**My format filter returns far fewer datasets than the portal shows.** It should not — that is exactly the trap this Actor fixes. If you passed a raw value through **Filter query** instead of using **Distribution formats**, you are hitting one of 403 spellings. Run the *Formats* dataset to see all of them.

**A licence filter returned nothing.** Licences live on the distributions, not the dataset. Use the **Licences** input, which matches there, not `license_id` through **Filter query** — that field is populated on 0.8% of the catalogue.

**A bounding-box filter isn't filtering.** There isn't one, deliberately: CKAN's `ext_bbox` returns the whole catalogue for any box on this portal. Use **Regional keys** or **Administrative level**.

**The run ended early saying it reached a charge ceiling.** That is Apify's per-run cost cap, derived from your remaining plan allowance. The rows already written are yours and the shortfall was not billed. Set **Max items** lower, or raise the allowance.

**A column preview came back `ok: false`.** Either the format is not tabular — in which case it was skipped and not billed — or the file needs authentication, or the link is dead. Turn on the link check to tell those apart.

**`hvdCategories` has a URI but `isHighValueDataset` is true with an empty array.** Some publishers put a German label in that field instead of the URI. The flag counts the label; the array keeps only resolvable URIs.

### FAQ

**Does it need a login or an API key?** No. Every surface used here is public and anonymous.

**Does it need a proxy?** No. 164 requests at up to 16 concurrent workers ran clean from a bare datacenter IP. Turn it on only if you are sweeping the whole catalogue repeatedly on a schedule.

**Can I get the whole catalogue in one run?** 156,461 datasets at 500 per page is 313 pages. Raise **Max catalogue pages**, sort by *Stable id order*, and watch your **Max items** so the run finishes inside your cost ceiling. Deep paging is safe here — 4,000 contiguous rows at three different sort orders returned zero duplicates.

**How current is it?** Live. GovData re-harvests continuously; 50,773 records changed in the last 7 days.

**Is the data in German?** The metadata is: titles, descriptions and keywords are German, because the datasets are. Category names come with both German and English titles, and the field names are English.

**What is DCAT-AP.de?** The German profile of the EU's DCAT-AP metadata standard. It is what every German portal publishes and what GovData harvests. This Actor flattens it into named fields instead of handing you the raw key/value list.

**What is an AGS?** The Amtlicher Gemeindeschlüssel, the official numeric key for every German municipality, district and Land. 34,436 datasets carry one, and you can filter on it.

**Why is `matchingDatasetCount` null on a directory row?** Because you ran without filters, so it would be identical to `datasetCount`. Set any filter and it fills in.

### Related actors

- [EU Open Data Portal Scraper](https://apify.com/parseforge/euopendata-datasets-scraper?fpr=vmoqkp) — the European Commission's catalogue, same DCAT lineage
- [Opendata Swiss Datasets Scraper](https://apify.com/parseforge/opendata-swiss-datasets-scraper?fpr=vmoqkp) — Switzerland's national portal
- [Luxembourg Open Data Scraper](https://apify.com/parseforge/luxembourg-open-data-scraper?fpr=vmoqkp) — data.public.lu
- [DWD Germany Weather Stations Scraper](https://apify.com/parseforge/dwd-germany-weather-stations-scraper?fpr=vmoqkp) — German weather observations at source

# Actor input Schema

## `dataset` (type: `string`):

Which surface of the GovData catalogue to return. 'Datasets' is one row per dataset with its distributions nested; 'Distributions' flattens to one row per file or service; 'Contact points' deduplicates the publishing bodies behind them. The remaining four are directories and each cost a single request.

## `searchQuery` (type: `string`):

Full-text search across titles, descriptions, tags and publishers, exactly as the portal's own search box works. German stemming applies, so 'Wasser' also matches 'Gewässer'. Leave empty for the whole catalogue.

## `titleContains` (type: `string`):

Restrict the search to dataset titles. Combined with Search query using AND.

## `descriptionContains` (type: `string`):

Restrict the search to dataset descriptions. Combined with Search query using AND.

## `organizations` (type: `array`):

Publishing organization slugs, e.g. berlin-open-data, open-data-bayern, transparenzportal-hamburg, gdi-de. Run the 'Organizations' dataset once to get the full list of 76 with their dataset counts. Multiple values are ORed.

## `categories` (type: `array`):

The 13 EU data themes. Multiple values are ORed.

## `tags` (type: `array`):

Exact keyword tags, e.g. opendata, bauleitplanung, vermessung, haushalt. Case-sensitive and matched exactly — run the 'Tags' dataset to see what exists and how many datasets each covers. Multiple values are ORed.

## `formats` (type: `array`):

Keep only datasets that publish at least one distribution in these formats, e.g. CSV, JSON, GEOJSON, XLSX, SHP, WFS, PARQUET. Each name is expanded to every spelling the catalogue actually uses — the 403 raw values include full EU file-type URIs, bare names and free text, and filtering on the bare name alone would miss 88% of the CSV datasets.

## `excludeFormats` (type: `array`):

Drop any dataset that publishes one of these formats. Applied after fetching, on the normalised short name (CSV, PDF, WMS, …).

## `datasetTypes` (type: `array`):

dct:type as DCAT-AP.de records it. Populated on about a quarter of the catalogue; leave empty unless you specifically need it.

## `hvdOnly` (type: `boolean`):

Keep only datasets flagged as EU High-Value Datasets under Implementing Regulation (EU) 2023/138 — the categories member states must publish openly and free of charge. 10,514 of the catalogue qualify.

## `hvdCategories` (type: `array`):

Narrow to specific HVD categories. Overrides the checkbox above. Measured counts: earth-environment 9,419 · geospatial 603 · mobility 313 · statistics 217 · meteorological 81 · companies 5.

## `administrativeLevels` (type: `array`):

Which tier of German government the dataset covers, from the DCAT-AP.de political geocoding vocabulary. Measured: municipality 24,910 · state 19,986 · district 4,089 · federal 3,789.

## `regionalKeys` (type: `array`):

Amtlicher Gemeindeschlüssel codes, e.g. 130030000000 for Rostock or 11 for Berlin as a state. Matches the dcat-ap.de regionalKey and stateKey URIs. Multiple values are ORed.

## `language` (type: `string`):

The declared language of the dataset record. German covers 93,655 records; English only 289, so this is mostly useful for excluding non-German material.

## `updateFrequencies` (type: `array`):

How often the publisher says the data is refreshed. Two vocabularies are in use across the catalogue for the same idea, and each option matches both.

## `publisherName` (type: `string`):

Exact publisher name as DCAT-AP.de records it, e.g. 'Ministerium für Bildung'. This is the actual publishing body, which is finer-grained than the CKAN organization — 76 organizations front many thousands of publishers.

## `sourcePortal` (type: `string`):

Substring of the regional portal GovData harvested the record from, e.g. berlin, hamburg, rlp. GovData is a harvesting catalogue and this is the trail back to the original.

## `onlyPublicAccess` (type: `boolean`):

Keep only records that declare the EU access-right value PUBLIC (28,951 datasets). Most publishers put free German prose in this field instead of the controlled value, so this narrows hard — leave it off unless you need the machine-readable declaration.

## `withSpatialCoverage` (type: `boolean`):

Keep only records carrying a spatial extent, so every row gets a bounding box and centroid. 72,621 datasets qualify.

## `minResources` (type: `integer`):

Drop datasets with fewer than this many distributions. Set to 1 to exclude the 239 metadata-only records.

## `modifiedAfter` (type: `string`):

ISO date or datetime, e.g. 2026-08-01. Together with the sort order, this is how you run the Actor as a change monitor.

## `modifiedBefore` (type: `string`):

ISO date or datetime.

## `createdAfter` (type: `string`):

ISO date or datetime. Filters on when the record first appeared on GovData, which is how you find genuinely new datasets rather than re-harvested ones.

## `createdBefore` (type: `string`):

ISO date or datetime.

## `temporalFrom` (type: `string`):

ISO date. Filters on the period the DATA covers, not on when the record was edited. A dataset matches when its coverage window overlaps yours.

## `temporalTo` (type: `string`):

ISO date. See above.

## `licenses` (type: `array`):

Keep only datasets whose distributions carry one of these licences — substring match against the licence URI, e.g. dl-by-de, cc-by, cc-zero, official-work. Applied after fetching, because the licence lives on the distribution: package-level licence is empty on 99.2% of the catalogue.

## `searchText` (type: `string`):

Plain substring filter over the finished row's title, description, publisher, tags and place name. Runs after fetching and before any optional block is bought, so filtering never costs you a link check.

## `filterQuery` (type: `string`):

Raw Solr fq appended to everything above, e.g. extras\_conforms\_to:\* or res\_url:*wfs*. Careful: an unknown field name returns zero rows rather than an error.

## `checkResourceLinks` (type: `boolean`):

HEAD every distribution URL and attach status, content type, byte size, redirect target and latency. GovData harvests metadata from ~100 regional portals and never re-verifies the links: measured 2026-08-27 across the full age range of the catalogue, 5.3% of distributions were dead 404s with nothing in the metadata to warn you. Billed per URL checked.

## `previewResourceColumns` (type: `boolean`):

Read the first 64 KB of each CSV, TSV, JSON, GeoJSON or XML distribution with a range request and return its actual column names, delimiter, encoding and a sample row. The catalogue tells you a dataset is 'a CSV'; it never tells you what is inside. Billed per distribution previewed.

## `onlyLiveResources` (type: `boolean`):

Requires the link check above. Discards datasets where every distribution failed, or distributions that failed individually. Dropped rows are not billed.

## `includeOrganizationProfile` (type: `boolean`):

Only for the 'Organizations' dataset. One extra request per organization returning what it actually publishes: its themes, its top formats and its top keywords, each with counts. Billed per organization profiled.

## `includeRawExtras` (type: `boolean`):

Attach the untouched CKAN extras key/value map alongside the parsed fields. Free — the data is already in the response — but it roughly doubles the row size.

## `sortBy` (type: `string`):

Only sort fields verified to actually change the order are offered. An unknown Solr sort field does not error on this API, it silently reverts to relevance.

## `directoryLimit` (type: `integer`):

How many rows to return for the Tags and Formats directories. The tag vocabulary has 97,863 entries and the long tail is mostly one-off keywords.

## `maxItems` (type: `integer`):

Hard ceiling on rows written, and therefore on cost. Leave empty for no ceiling.

## `maxPages` (type: `integer`):

Each page is 500 datasets and bills one catalogue-page event. 20 pages covers 10,000 datasets.

## `concurrency` (type: `integer`):

Parallel requests, 1 to 12. Measured on a bare datacenter IP: 8 workers moved 525 dataset rows a second with zero failures and no throttling. Above 8 the per-request overhead starts to dominate.

## `timeoutSecs` (type: `integer`):

Soft wall-clock budget. The run stops issuing requests and flushes what it has. The platform's own run timeout still applies and whichever is nearer wins.

## `proxyConfiguration` (type: `object`):

Off by default and not needed: 84 requests at up to 16 concurrent workers ran clean from a bare datacenter IP with no throttling. Turn it on only if you are sweeping the whole catalogue repeatedly on a schedule.

## Actor input object example

```json
{
  "dataset": "datasets",
  "hvdOnly": false,
  "language": "",
  "onlyPublicAccess": false,
  "withSpatialCoverage": false,
  "checkResourceLinks": false,
  "previewResourceColumns": false,
  "onlyLiveResources": false,
  "includeOrganizationProfile": false,
  "includeRawExtras": false,
  "sortBy": "modified-desc",
  "directoryLimit": 500,
  "maxPages": 20,
  "concurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/govdata-germany-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchQuery": "" }

# Run the Actor and wait for it to finish
run = client.actor("parseforge/govdata-germany-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": ""
}' |
apify call parseforge/govdata-germany-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/govdata-germany-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8cdbNVIDLpkl3SPeU/builds/ueU880K39RPe2lkWo/openapi.json