OParl Council Information Extractor

Point at any German OParl endpoint and get clean, flat rows for a municipality's papers, meetings, committees and people — plus the lossless raw JSON of every object. One actor, every council-information system.

OParl (schema.oparl.org) is the open standard for the German municipal Ratsinformationssystem (council information system) — the REST/JSON API that publishes a city's or municipality's papers (Drucksachen), meetings (Sitzungen), committees (Gremien) and council members. The entire point of OParl is that the data graph is identical across every vendor implementation:

System → Body → { Paper, Meeting, Organization, Person, LegislativeTerm }

So this single actor speaks to every OParl server in Germany — STERNBERG SD.NET, more-rubin (gremien.info), CC e-gov / Session, regisafe, ALLRIS and the rest — with no per-city scraper and no host enumeration. Just point it at any System or Body URL. A live registry of public endpoints is at dev.oparl.org/api/endpoints.

What you can extract

Pick a mode and point sources at one or more OParl base URLs:

Mode	What you get (one row per…)
papers (default)	a Paper / Drucksache — reference, title, date, paperType, main-file URL, consultation count, body, web link + raw JSON
meetings	a Meeting / Sitzung — name, start, end, cancelled, organization(s), location, body
organizations	a committee / department — name, shortName, organizationType, classification, start/end date
persons	a council member — name, family/given name, form of address, gender
discovery	a census row per Body — the System identity + every object-list URL, to inventory an endpoint first

Why one actor covers every vendor

OParl's wire format varies in exactly the ways a naive client trips on — and this runner handles all of them:

• Object-list URLs differ by version and vendor. A v1.1 STERNBERG Body exposes …/webservice/oparl/v1.1/body/1/paper; a v1.0 CC e-gov Body exposes …/oi/oparl/1.0/papers.asp?body=1; more-rubin exposes …/oparl/body/<id>/paper. The list URL for every mode is read from the Body object's own link field — never hard-coded — so any layout works.
• Pagination differs by vendor. STERNBERG and more-rubin return a pagination.totalElements block and a links.next cursor; CC e-gov returns no totalElements, only a links.next cursor URL. The paginator follows links.next (the spec's cursor — works everywhere) and falls back to deriving the next page from pagination only when a server omits the link. A totalElements-only client would silently stop after page 1 on every CC e-gov host; this one does not.
• Soft-disabled endpoints return HTTP 200 + an OParl Error body ("OParl is not active."). Those are detected and skipped with a warning — never emitted as a record.
• Resilience. A browser User-Agent is sent, transient failures are retried with backoff, and an unreachable source is skipped with a warning. If every source is skipped the run fails fast; a reachable endpoint with zero public objects yields zero rows and a clean exit. Output is never fabricated.

Input

Field	Description
sources	One or more OParl base URLs (a System, a Body, or a body-list URL). Leave empty to run against a built-in set of verified endpoints.
mode	papers (default) / meetings / organizations / persons / discovery.
body	Optional — restrict to Bodies whose id/name/shortName contains this text.
modifiedSince	Optional incremental harvest — only objects modified on/after this ISO 8601 timestamp (sent as OParl ?modified_since=).
maxRecords	Global cap on total rows across all sources and pages (default 1000).
bearer / extraHeaders	Optional auth for gated deployments. Never required for public systems; never logged.

Output

One flat, fully-typed row per object. Every optional field is nullable, so rows from different vendors and modes share one stable schema. Provenance columns (_sourceSystem, _vendor, _oparlVersion, _body, _bodyName) and the lossless _raw JSON are attached to every row.

{
  "_mode": "papers",
  "_vendor": "STERNBERG SD.NET",
  "_oparlVersion": "1.1",
  "_bodyName": "Gemeinde Aarbergen",
  "reference": "VL-22/2013",
  "name": "Voranschlag über die Einnahmen und Ausgaben …",
  "date": "2013-10-10",
  "paperType": "Beschlussvorlage Gemeindevertretung",
  "mainFileUrl": "https://…/files/…/Beschlussvorlage_….pdf",
  "consultationCount": 1,
  "id": "https://…/body/1/paper/93",
  "web": "https://…/vorgang/?__=…"
}

Pricing

Pay-per-event: one record charge per emitted row. No subscription — you pay only for what you harvest.

Who it's for

Civic-tech and govtech teams, journalists and researchers tracking municipal decisions, transparency / open-data projects, and anyone building on Germany's council-information infrastructure without writing a scraper per city.