Buildbot Data Extractor

Point at any Buildbot CI master and get clean, structured rows — one per builder, build, or worker. Buildbot is the self-hosted continuous-integration engine behind CPython, WebKit, LLVM, Mozilla and GNOME, and countless corporate and university deployments. Every Buildbot master exposes the identical data API v2 as open, anonymous-read JSON, so this one actor works against all of them — no per-server scraper, no login.

What it does

Three modes, chosen by the Run mode field:

• builders (default) — the master's builder (build-configuration) inventory, one row per builder: name, tags, description, project id, master ids.
• builds — recent builds, newest first: build number, result code and a human status label (success / warnings / failure / skipped / exception / retry / cancelled), the live state string, start/complete timestamps (ISO-8601), and the worker that ran it.
• workers — the master's worker/agent inventory, one row per worker: name, connected/paused state, host description and worker version.

Point it at several masters at once and it runs each in turn; a URL that isn't a Buildbot master (a landing page, or a host that never had the data API) is skipped with a warning instead of failing the whole run.

Example input

{
  "sources": ["https://buildbot.python.org/all"],
  "mode": "builders",
  "maxRecords": 100
}

Recent builds on WebKit, scoped to one builder family:

{
  "sources": ["https://build.webkit.org"],
  "mode": "builds",
  "builderFilter": "Debug",
  "maxRecords": 200
}

Other ready-to-run masters: https://lab.llvm.org/buildbot (LLVM), https://buildbot.buildbot.net (Buildbot itself). Any self-hosted master works too — paste its base URL. The API mount path varies: CPython mounts the multi-project API at /all/api/v2, most others sit at /api/v2 on the root; paste either form and the mount is auto-detected.

Output

One flat row per record. Common columns: _type (builder / build / worker), _source (the master), _software (always buildbot). Builder rows add builderid, name, tags, description. Build rows add buildid, number, complete, results (raw code) + status (label), stateString, startedAt, completeAt, workerid. Worker rows add connected/paused flags and the worker host/version. Every row also carries a lossless _raw copy of the original API entry (capped for size).

Modes and pagination

• builderFilter is a case-insensitive substring matched against a builder's name or tags. In builds mode, when it resolves to exactly one builder the query is scoped server-side (fast); when it matches several, only builds from those builders are kept.
• maxRecords is a global cap across all sources — a cheap, deterministic sample. Large masters have millions of builds, so a cap is recommended in builds mode; the builder and worker inventories are naturally bounded.

Authentication

Public masters need none. For a private/registration-required master, supply a bearer token or extraHeaders (e.g. a Cookie) — these are sent on every request and are never logged.

Pricing

Pay-per-event: one charge per emitted record (builder / build / worker). A run's cost is exactly the number of rows returned, so maxRecords bounds it precisely.