# Buildbot Data Extractor (`datamule/buildbot-data-extractor`) Actor Point at any Buildbot CI master (CPython, WebKit, LLVM, GNOME, self-hosted) and get one structured row per builder, build, or worker: names, tags, build results and status, timestamps, and worker inventory. One actor for every Buildbot data API v2 server. - **URL**: https://apify.com/datamule/buildbot-data-extractor.md - **Developed by:** [Datamule](https://apify.com/datamule) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.35 / 1,000 records This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Buildbot Data Extractor Point at **any** Buildbot CI master and get clean, structured rows — one per builder, build, or worker. Buildbot is the self-hosted continuous-integration engine behind **CPython, WebKit, LLVM, Mozilla and GNOME**, and countless corporate and university deployments. Every Buildbot master exposes the identical **data API v2** as open, anonymous-read JSON, so this one actor works against all of them — no per-server scraper, no login. ### What it does Three modes, chosen by the **Run mode** field: - **builders** (default) — the master's builder (build-configuration) inventory, one row per builder: name, tags, description, project id, master ids. - **builds** — recent builds, newest first: build number, result code and a human status label (success / warnings / failure / skipped / exception / retry / cancelled), the live state string, start/complete timestamps (ISO-8601), and the worker that ran it. - **workers** — the master's worker/agent inventory, one row per worker: name, connected/paused state, host description and worker version. Point it at several masters at once and it runs each in turn; a URL that isn't a Buildbot master (a landing page, or a host that never had the data API) is skipped with a warning instead of failing the whole run. ### Example input ```json { "sources": ["https://buildbot.python.org/all"], "mode": "builders", "maxRecords": 100 } ```` Recent builds on WebKit, scoped to one builder family: ```json { "sources": ["https://build.webkit.org"], "mode": "builds", "builderFilter": "Debug", "maxRecords": 200 } ``` Other ready-to-run masters: `https://lab.llvm.org/buildbot` (LLVM), `https://buildbot.buildbot.net` (Buildbot itself). Any self-hosted master works too — paste its base URL. The API mount path varies: CPython mounts the multi-project API at `/all/api/v2`, most others sit at `/api/v2` on the root; paste either form and the mount is auto-detected. ### Output One flat row per record. Common columns: `_type` (builder / build / worker), `_source` (the master), `_software` (always `buildbot`). Builder rows add `builderid`, `name`, `tags`, `description`. Build rows add `buildid`, `number`, `complete`, `results` (raw code) + `status` (label), `stateString`, `startedAt`, `completeAt`, `workerid`. Worker rows add connected/paused flags and the worker host/version. Every row also carries a lossless `_raw` copy of the original API entry (capped for size). ### Modes and pagination - `builderFilter` is a case-insensitive substring matched against a builder's name or tags. In **builds** mode, when it resolves to exactly one builder the query is scoped server-side (fast); when it matches several, only builds from those builders are kept. - `maxRecords` is a **global** cap across all sources — a cheap, deterministic sample. Large masters have millions of builds, so a cap is recommended in builds mode; the builder and worker inventories are naturally bounded. ### Authentication Public masters need none. For a private/registration-required master, supply a `bearer` token or `extraHeaders` (e.g. a Cookie) — these are sent on every request and are **never logged**. ### Pricing Pay-per-event: one charge per emitted record (builder / build / worker). A run's cost is exactly the number of rows returned, so `maxRecords` bounds it precisely. # Actor input Schema ## `sources` (type: `array`): One or more Buildbot CI master base URLs. Every Buildbot master exposes the identical Buildbot data API v2, so one actor works against any of them: https://buildbot.python.org/all (CPython), https://build.webkit.org (WebKit), https://lab.llvm.org/buildbot (LLVM), https://buildbot.buildbot.net (Buildbot itself), or any self-hosted / corporate deployment. The API mount path varies — CPython uses /all/api/v2 while others sit at /api/v2 on the root; paste either the base or the /all form and the mount is auto-detected. Anonymous read is on for public masters, so no login is needed. ## `mode` (type: `string`): What to extract from each master. builders (default) = one row per builder (build configuration): name, tags, description. builds = recent builds newest-first: build number, result status, state, timestamps, worker. workers = the master's worker/agent inventory, one row per worker. ## `builderFilter` (type: `string`): Optional case-insensitive substring matched against a builder's name or any of its tags. In builders mode it filters which builders are emitted. In builds mode, if it resolves to exactly one builder the builds query is scoped server-side to that builder (fast); if it matches several, only builds from the matching builders are kept. Leave empty for all builders / all recent builds. Ignored in workers mode. ## `maxRecords` (type: `integer`): A GLOBAL cap on the number of rows to emit across all sources (each row is one builder/build/worker and one billable event). Pagination stops as soon as the cap is reached, so a small value is a cheap, deterministic sample. Leave empty to fetch every record — a broad builds query on a large master (CPython/WebKit have millions of builds) can return a lot; the builders and workers inventories are bounded (hundreds to low thousands). ## `bearer` (type: `string`): Optional access token for a private / registration-required master (sent as Authorization: Bearer ...). Not required for public masters. Never logged. ## `extraHeaders` (type: `object`): Optional extra HTTP headers as a JSON object, e.g. {"x-api-key": "..."} or a raw Cookie for a gated master. Not required for public data. Header values are never logged. ## Actor input object example ```json { "sources": [ "https://buildbot.python.org/all" ], "mode": "builders" } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sources": [ "https://buildbot.python.org/all" ] }; // Run the Actor and wait for it to finish const run = await client.actor("datamule/buildbot-data-extractor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sources": ["https://buildbot.python.org/all"] } # Run the Actor and wait for it to finish run = client.actor("datamule/buildbot-data-extractor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sources": [ "https://buildbot.python.org/all" ] }' | apify call datamule/buildbot-data-extractor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=datamule/buildbot-data-extractor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Buildbot Data Extractor", "description": "Point at any Buildbot CI master (CPython, WebKit, LLVM, GNOME, self-hosted) and get one structured row per builder, build, or worker: names, tags, build results and status, timestamps, and worker inventory. One actor for every Buildbot data API v2 server.", "version": "0.1", "x-build-id": "pbfXtBIuU2BT9ohLq" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/datamule~buildbot-data-extractor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-datamule-buildbot-data-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/datamule~buildbot-data-extractor/runs": { "post": { "operationId": "runs-sync-datamule-buildbot-data-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/datamule~buildbot-data-extractor/run-sync": { "post": { "operationId": "run-sync-datamule-buildbot-data-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sources" ], "properties": { "sources": { "title": "Buildbot master URLs", "type": "array", "description": "One or more Buildbot CI master base URLs. Every Buildbot master exposes the identical Buildbot data API v2, so one actor works against any of them: https://buildbot.python.org/all (CPython), https://build.webkit.org (WebKit), https://lab.llvm.org/buildbot (LLVM), https://buildbot.buildbot.net (Buildbot itself), or any self-hosted / corporate deployment. The API mount path varies — CPython uses /all/api/v2 while others sit at /api/v2 on the root; paste either the base or the /all form and the mount is auto-detected. Anonymous read is on for public masters, so no login is needed.", "items": { "type": "string" } }, "mode": { "title": "Run mode", "enum": [ "builders", "builds", "workers" ], "type": "string", "description": "What to extract from each master. builders (default) = one row per builder (build configuration): name, tags, description. builds = recent builds newest-first: build number, result status, state, timestamps, worker. workers = the master's worker/agent inventory, one row per worker.", "default": "builders" }, "builderFilter": { "title": "Builder filter (MODE: builders / builds)", "type": "string", "description": "Optional case-insensitive substring matched against a builder's name or any of its tags. In builders mode it filters which builders are emitted. In builds mode, if it resolves to exactly one builder the builds query is scoped server-side to that builder (fast); if it matches several, only builds from the matching builders are kept. Leave empty for all builders / all recent builds. Ignored in workers mode." }, "maxRecords": { "title": "Max records (global cap)", "minimum": 1, "type": "integer", "description": "A GLOBAL cap on the number of rows to emit across all sources (each row is one builder/build/worker and one billable event). Pagination stops as soon as the cap is reached, so a small value is a cheap, deterministic sample. Leave empty to fetch every record — a broad builds query on a large master (CPython/WebKit have millions of builds) can return a lot; the builders and workers inventories are bounded (hundreds to low thousands)." }, "bearer": { "title": "Bearer token", "type": "string", "description": "Optional access token for a private / registration-required master (sent as Authorization: Bearer ...). Not required for public masters. Never logged." }, "extraHeaders": { "title": "Extra request headers", "type": "object", "description": "Optional extra HTTP headers as a JSON object, e.g. {\"x-api-key\": \"...\"} or a raw Cookie for a gated master. Not required for public data. Header values are never logged." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```