# OTTL Transformation Generator & Simulator (`michiganpublictech/ottl-studio`) Actor

Plain-English requests → OpenTelemetry OTTL (transform/filter processors), an OTTL linter with 56 rules, and a simulator that shows before/after diffs on sample traces, logs and metrics. Deterministic, no LLM; Standby HTTP for agents.

- **URL**: https://apify.com/michiganpublictech/ottl-studio.md
- **Developed by:** [Justin Hagerty](https://apify.com/michiganpublictech) (community)
- **Categories:** Developer tools, AI
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $100.00 / 1,000 ottl request processeds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## OTTL Transformation Generator & Simulator

**Describe the telemetry change you want in plain English and get a ready-to-paste OpenTelemetry Collector `transform` / `filter` processor in OTTL, linted by 66 rules and executed against sample traces, logs and metrics with before/after diffs — before it ever touches production.**

The [OpenTelemetry Transformation Language (OTTL)](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl) is how the Collector redacts PII, drops noisy spans, renames attributes and reshapes metrics. It is also easy to get subtly wrong: a regex written for Python instead of RE2, a `$` that YAML expansion eats, a statement in the wrong context, a `delete_key` that runs before the `set` that needed the value, a filter condition that silently drops 90% of your traces, `error_mode: propagate` dropping whole batches. None of these fail loudly.

This Actor is a **deterministic OTTL studio**: it recognises 61 common transformation intents, generates OTTL in the syntax your Collector version understands, validates both generated and hand-written OTTL, and **simulates** the statements on realistic telemetry so you see exactly what changes, what gets dropped and what breaks. No LLM, no network calls, same answer every time — as a normal Apify run, in CI, or over **Standby HTTP** for AI coding agents and MCP clients.

***

### Why use it

- **Plain English → correct OTTL.** "Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans" becomes a `transform` processor on the right contexts (resource, span, span event, log, data point), a `filter` processor that keeps failing probes, `$$`-escaped YAML and the pipeline order to use.
- **Lint OTTL you already have.** Paste statements or a whole Collector config: syntax errors with line/column, unknown functions and wrong arity, paths that don't exist in the context, RE2-incompatible regexes, unescaped `$`, ordering bugs (read after delete), always-true filters, legacy filter config, `error_mode` risks and version incompatibilities.
- **See the effect before deploying.** The simulator runs the statements on your sample telemetry (or bundled realistic samples) and returns per-item diffs, per-statement hit counts, byte-size reduction and danger findings: filters that drop too much, `service.name` or trace context removed, dead rules, empty bodies, runtime errors.
- **Honest about limits.** Phrases it doesn't understand are listed with the closest supported intents, functions it can't simulate are reported as *not simulated* — nothing is guessed or faked.

### Who it's for

- **Platform and observability engineers** writing redaction, cost-reduction and normalisation rules for a shared Collector fleet.
- **Teams cutting telemetry bills** who want to know how much a filter drops before rolling it out.
- **AI coding agents and MCP clients** that edit Collector configs: generate, validate and simulate in a loop through one cheap HTTP call.
- **CI pipelines** that lint every change to `otel-collector.yaml` (`failOnSeverity`, SARIF) — a fast pre-check before `otelcol validate`.

### How to use

#### In Apify Console

1. Write what you want in **Requirements** (the prefill is a working example), and/or paste OTTL into **Statements** or a full **Collector config**.
2. Optionally paste **Sample telemetry** (OTLP/JSON) — otherwise the bundled samples (HTTP/DB/health-check spans, logs with PII, metrics) are used.
3. Set your **Collector version** so the syntax matches your release, then click **Start**. The **Output** tab has four views: **Overview**, **Generated config**, **Findings** and **Simulation**.

#### Via the API (curl)

```bash
curl -s -X POST \
  "https://api.apify.com/v2/acts/michiganpublictech~ottl-studio/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"requirements": "Redact emails in logs; drop debug logs; rename `user_id` to `user.id`",
       "collectorVersion": "0.150.0"}' \
  | jq -r '.[0].data.generated.collector_yaml'
```

#### Python (apify-client)

```python
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("michiganpublictech/ottl-studio").call(
    run_input={
        "mode": "validate",
        "collectorConfig": open("otel-collector.yaml").read(),
        "failOnSeverity": "high",
    }
)
for report in client.dataset(run["defaultDatasetId"]).iterate_items():
    for f in report["findings"]:
        print(f["severity"], f["rule_id"], f["location"], f["title"])
```

#### CI: lint the Collector config on every change

```yaml
- name: Lint OTTL
  run: |
    jq -n --rawfile cfg otel-collector.yaml '{mode: "all", collectorConfig: $cfg, failOnSeverity: "high"}' > input.json
    curl -sf -X POST "https://api.apify.com/v2/acts/michiganpublictech~ottl-studio/runs?token=${{ secrets.APIFY_TOKEN }}&waitForFinish=300" \
      -H "Content-Type: application/json" -d @input.json | jq -e '.data.status == "SUCCEEDED"'
```

With `failOnSeverity` the run fails when any finding (lint or simulation) is at or above that severity. `REPORT.sarif` in the run's key-value store can be uploaded to GitHub code scanning.

### Standby mode: HTTP API for AI agents and MCP clients

The Actor supports [Actor Standby](https://docs.apify.com/platform/actors/running/standby): a warm instance behind an HTTPS endpoint answers in milliseconds with no container start-up. **Charging is identical to batch runs**, and nothing is written to a dataset — the report comes back in the HTTP response.

```bash
BASE=https://michiganpublictech--ottl-studio.apify.actor
AUTH="Authorization: Bearer $APIFY_TOKEN"

## Free: capabilities, intent catalog, function catalog, rule catalog
curl -s "$BASE/" -H "$AUTH"
curl -s "$BASE/intents" -H "$AUTH"
curl -s "$BASE/functions?verbose=1" -H "$AUTH"
curl -s "$BASE/rules" -H "$AUTH"

## Generate (no simulation → only transform-generated is charged)
curl -s -X POST "$BASE/generate" -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"requirements": "Hash user IDs with SHA256 and drop spans for /internal/*", "collectorVersion": "0.110.0"}'

## Validate your own OTTL
curl -s -X POST "$BASE/validate" -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"statementContext": "span", "statements": ["delete_key(attributes, \"user.email\")",
       "set(attributes[\"email_domain\"], Split(attributes[\"user.email\"], \"@\")[1])"]}'
## → OTTL-SEM-004: user.email is read after an earlier statement deleted it

## Simulate on your telemetry (statements, a collectorConfig or requirements)
curl -s -X POST "$BASE/simulate" -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"requirements": "Drop successful health-check spans",
       "sampleTelemetry": {"spans": [{"name": "GET /healthz", "attributes": {"http.route": "/healthz"}},
                                     {"name": "GET /healthz", "status": {"code": "error"}, "attributes": {"http.route": "/healthz"}}]}}'
```

Endpoints: `GET /` (help; the platform readiness probe gets a plain `ok`), `GET /intents`, `GET /functions`, `GET /rules`, `POST /generate`, `POST /validate`, `POST /simulate`, `POST /run` (alias `POST /`; full Actor input, `mode` defaults to `all`). Options set on the Standby run's input (`collectorVersion`, `syntaxStyle`, `processorName`, `signals`, `errorMode`, …) act as defaults; the request body overrides them. Bodies are limited to 4 MB. Invalid input returns `400`; a request that cannot be paid for (spending limit) returns `402`; requests whose requirements are not understood return `status: "failed"` and are free.

#### From an MCP client

The [Apify MCP server](https://docs.apify.com/platform/integrations/mcp) exposes the Actor as a tool (input schema = tool definition):

```json
{
  "mcpServers": {
    "ottl-studio": {
      "url": "https://mcp.apify.com?tools=michiganpublictech/ottl-studio",
      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
    }
  }
}
```

Suggested agent instruction: *"Before editing transform/filter processors in the Collector config, describe the change to the ottl-studio tool (or send the edited config with mode=all). Fix every high/critical finding and check `data.simulation.stats` for unexpected drops before committing."* Agents that can make plain HTTP calls can use the Standby endpoints directly for the lowest latency.

### Supported intents

61 deterministic intents. Each understands variations of the phrasing (synonyms, quoted or dotted attribute names, aliases such as "pod UIDs" → `k8s.pod.uid`, signal scoping such as "in logs", numbers, routes, `/regex/`). Several requests can be combined in one sentence, separated by commas, `and`, `;`, new lines or bullet points. `GET /intents` returns the same list with more examples.

| Category | Intent | What it does | Processor | Example phrasing |
|---|---|---|---|---|
| redaction | `drop_sensitive_keys` | Delete secret-looking attributes | transform | Remove sensitive attributes like passwords and API keys |
| redaction | `drop_stack_traces` | Remove exception stack traces | transform | Drop exception stack traces |
| redaction | `hash_attributes` | Hash attribute values (SHA256) | transform | Hash user IDs with SHA256 |
| redaction | `mask_attribute_value` | Mask an attribute's value | transform | Mask the value of `user.email` |
| redaction | `redact_bearer_tokens` | Redact bearer tokens | transform | Redact bearer tokens |
| redaction | `redact_credit_cards` | Redact credit-card numbers | transform | Redact credit card numbers |
| redaction | `redact_emails` | Redact e-mail addresses | transform | Redact user email addresses |
| redaction | `redact_ip_addresses` | Redact IPv4 addresses | transform | Mask client IP addresses |
| redaction | `redact_jwts` | Redact JWTs | transform | Redact JWTs |
| redaction | `redact_phone_numbers` | Redact phone numbers | transform | Redact phone numbers |
| redaction | `redact_ssns` | Redact US social security numbers | transform | Redact SSNs |
| redaction | `strip_query_strings` | Strip URL query strings | transform | Strip query strings from url.full |
| redaction | `strip_url_credentials` | Strip credentials embedded in URLs | transform | Remove passwords embedded in URLs |
| attributes | `add_environment` | Tag telemetry with the deployment environment | transform | Add environment=production to everything |
| attributes | `copy_attribute` | Copy an attribute to a new key | transform | Copy `http.route` to `route.template` |
| attributes | `copy_resource_attribute` | Copy resource attributes onto items | transform | Copy resource attribute service.version to span attributes |
| attributes | `default_service_name` | Default service.name when missing | transform | Default service.name to "legacy-batch" when missing |
| attributes | `delete_attributes` | Delete attributes | transform | Remove Kubernetes pod UIDs |
| attributes | `delete_matching_attributes` | Delete attributes matching a pattern | transform | Remove all attributes starting with "http.request.header." |
| attributes | `keep_matching_attributes` | Keep only attributes matching a pattern | transform | Keep only attributes starting with "http." |
| attributes | `keep_only_attributes` | Keep only listed attributes | transform | Keep only `http.route` and `http.response.status_code` on spans |
| attributes | `migrate_semconv` | Migrate legacy semantic conventions | transform | Migrate legacy HTTP semantic conventions to the stable names |
| attributes | `rename_attribute` | Rename an attribute | transform | Rename `user_id` to `user.id` |
| attributes | `set_attribute` | Set an attribute to a static value | transform | Set `team` = "payments" on all spans |
| attributes | `set_attribute_from_env` | Set an attribute from an environment variable | transform | Set `k8s.cluster.name` from env var CLUSTER_NAME |
| attributes | `set_attribute_if_missing` | Set an attribute only when missing | transform | Add `team` = "payments" if missing |
| size | `limit_attribute_count` | Limit the number of attributes | transform | Limit spans to 64 attributes |
| size | `truncate_attribute_values` | Truncate attribute values | transform | Truncate attribute values to 256 characters |
| size | `truncate_log_body` | Truncate log bodies | transform | Truncate log bodies to 1024 characters |
| normalization | `flatten_attributes` | Flatten nested maps | transform | Flatten nested attributes |
| normalization | `lowercase_attribute` | Lowercase attribute values | transform | Lowercase `http.request.method` |
| normalization | `normalize_http_route` | Normalize http.route | transform | Normalize http.route, replacing numeric IDs with {id} |
| normalization | `normalize_span_names` | Template IDs in span names | transform | Replace IDs and UUIDs in span names with placeholders |
| normalization | `uppercase_attribute` | Uppercase attribute values | transform | Uppercase `http.request.method` |
| parsing | `extract_with_regex` | Extract fields with a regex | transform | Extract fields from the log body using regex /user=(?P<user>\w+)/ |
| parsing | `parse_json_body` | Parse JSON log bodies into attributes | transform | Parse JSON log bodies into attributes |
| parsing | `parse_key_value_body` | Parse key=value log bodies | transform | Parse key=value pairs from log bodies into attributes |
| parsing | `severity_from_body` | Derive log severity from the body text | transform | Set log severity from the body text |
| filtering | `drop_by_namespace` | Drop telemetry from a Kubernetes namespace | filter | Drop everything from namespace kube-system |
| filtering | `drop_by_service` | Drop telemetry from a service | filter | Drop all telemetry from service "load-generator" |
| filtering | `drop_datapoints_by_attribute` | Drop metric data points by attribute | filter | Drop datapoints where `http.route` = "/healthz" |
| filtering | `drop_debug_logs` | Drop DEBUG/TRACE logs | filter | Drop debug logs |
| filtering | `drop_health_checks` | Drop health-check spans | filter | Drop health check spans |
| filtering | `drop_logs_below_severity` | Drop logs below a severity | filter | Drop logs below WARN |
| filtering | `drop_logs_matching` | Drop logs whose body matches text | filter | Drop logs containing "connection reset by peer" |
| filtering | `drop_metrics_by_name` | Drop metrics by name | filter | Drop the metric `process.runtime.cpython.gc_count` |
| filtering | `drop_metrics_by_prefix` | Drop metrics by name prefix/regex | filter | Drop metrics starting with "debug." |
| filtering | `drop_short_spans` | Drop very short spans | filter | Drop spans shorter than 1ms |
| filtering | `drop_span_events_by_name` | Drop span events by name | filter | Drop span events named "message" |
| filtering | `drop_spans_by_attribute` | Drop spans by attribute value | filter | Drop spans where `db.system` = "redis" |
| filtering | `drop_spans_by_kind` | Drop spans of a kind | filter | Drop internal spans |
| filtering | `drop_spans_by_name` | Drop spans by name | filter | Drop spans named "GET /favicon.ico" |
| filtering | `drop_spans_by_route` | Drop spans for a route | filter | Drop spans for /internal/\* |
| filtering | `drop_successful_health_checks` | Drop successful health-check spans | filter | Drop successful health-check spans |
| filtering | `keep_only_error_logs` | Keep only ERROR+ logs | filter | Keep only error logs |
| filtering | `keep_only_error_spans` | Keep only error spans | filter | Keep only error spans |
| traces | `span_status_from_http` | Set span status ERROR from HTTP status | transform | Mark spans with HTTP 5xx responses as errors |
| metrics | `convert_gauge_to_sum` | Convert gauges to sums | transform | Convert gauge `jobs.done` to a sum |
| metrics | `convert_sum_to_gauge` | Convert sums to gauges | transform | Convert sum `queue.depth` to a gauge |
| metrics | `rename_metric` | Rename a metric | transform | Rename metric `old.name` to `new.name` |
| metrics | `scale_metric_unit` | Convert a metric's unit | transform | Convert `db.client.latency` from ms to s |

Something missing? Phrases that match nothing come back under `data.unrecognized` with the three closest intents; phrases that match an intent but lack a required detail (e.g. "rename the attribute" without names) say what is missing.

### Collector versions and syntax style

| Collector | Generated syntax | Notes |
|---|---|---|
| < 0.120 (default target 0.110.0) | **Grouped**: `trace_statements: [{context: span, statements: [...]}]` | Works on every OTTL-capable release. |
| ≥ 0.120 | **Flat**: `trace_statements: ['set(span.attributes["x"], ...)']` with context-prefixed paths (`resource.`, `span.`, `spanevent.`, `log.`, `metric.`, `datapoint.`) | Context inferred per statement. Override with `syntaxStyle`. |

Filter conditions are always emitted in the classic per-context blocks (`traces.span`, `logs.log_record`, `metrics.datapoint`, …), which all OTTL-capable Collectors accept. Regex `$` is written as `$$` so Collector environment expansion leaves it alone.

**Verified with real Collectors:** the output of every intent example (all 61 intents, 96 examples) was checked with `otelcol-contrib validate` — grouped style on **0.115**, **0.148** and **0.150.1**, flat style on **0.148** and **0.150.1**. Flat transform lists are rejected by 0.115 ("expected a map, got 'string'"), which is why older versions get the grouped style. The 0.120 cut-over and the other entries of the bundled version table (`error_mode` ~0.71, `silent` ~0.95, named arguments ~0.100, function `since` hints) are best-effort and labelled as such in the output; confirm against the transformprocessor README of your exact release.

### Validation rules

**66 rules**: 56 lint rules and 10 simulation rules. Full catalog with recommendations: **[RULES.md](RULES.md)**.

| Family | Rules | What they catch |
|---|---|---|
| `OTTL-SYN` syntax | 4 | Statements that don't parse (with line:column and hints for `'single quotes'`, `&&`, `!`), conditions where a statement is required and vice versa, empty statements |
| `OTTL-FN` functions | 12 | Unknown functions, wrong argument count or type, converter used as a statement / editor inside an expression, non-writable targets, unknown named arguments, invalid options and enums, deprecated functions, functions that may need a newer Collector or are not available in the context |
| `OTTL-PATH` paths | 8 | Unknown paths for the context, paths from another context, writes to read-only fields, `attributes.key` instead of `attributes["key"]`, parent-context writes from a child context, context-prefixed paths on older Collectors, legacy semconv attributes in conditions, contexts that cannot be inferred |
| `OTTL-RE` regex | 8 | Regexes that don't compile or use syntax RE2 lacks (look-around, backreferences), `\1` instead of `$1`, `ExtractPatterns` without named groups, invalid Go string escapes (`"\d"`), unanchored or unescaped key patterns, redundant `.*` |
| `OTTL-SEM` semantics & ordering | 13 | Read after delete, values overwritten before use, deleting `service.name`, unconditional statements and overwrites, copies that keep the original, duplicates, constant `where` clauses, typed fields compared with or set to the wrong type, non-positive limits, `keep_keys` with an empty list, body parsers without a type guard |
| `OTTL-CFG` configuration | 11 | `error_mode: propagate` / invalid `error_mode`, processors not used in any pipeline or without statements for its signal, transform/filter after `batch`, always-true (drops everything) and always-false filters, invalid statement groups, groups under the wrong signal, unescaped `$`, legacy include/exclude filters |
| `OTTL-SIM` simulation | 10 | Filters dropping at least `dropWarningPercent` of items, `service.name` or trace context removed, overwritten values, empty bodies or span names, dead rules, patterns that changed every item, runtime errors, read-after-delete observed at runtime, statements that could not be simulated, filters that dropped nothing |

### Simulation

Statements run in pipeline order (from your `collectorConfig`, otherwise filter → transform, or transform → filter when the filter reads fields the transform writes) on a copy of the telemetry, following the Collector's semantics:

- **Transform:** statements per context in order; `where` per statement; group-level `conditions` (OR); `nil` values make `set` a no-op; resource/scope statements apply once per resource/scope.
- **Filter:** conditions are OR-ed; matching spans, span events, log records, metrics or data points are dropped, and metrics left without data points are removed — as the filter processor does.
- **`error_mode`:** `ignore`/`silent` skip the failing statement for that item; `propagate` rejects the signal's whole batch (reported as a finding).
- **Output:** `data.simulation` holds per-signal `items_in/out/dropped`, estimated bytes before/after and reduction %, attributes added/removed, values changed, up to `maxDiffItems` item diffs (`removed`, `added`, `changed` with before/after, `dropped`), and per-statement stats (`evaluated`, `matched`, `changed`, `errors`).

**Simulated (48 functions):** editors `set`, `delete_key`, `delete_matching_keys`, `keep_keys`, `keep_matching_keys`, `limit`, `merge_maps`, `replace_pattern`, `replace_all_patterns`, `replace_match`, `replace_all_matches`, `truncate_all`, `flatten`, `append`, `convert_sum_to_gauge`, `convert_gauge_to_sum`, `scale_metric`; converters `Concat`, `ConvertCase`, `Double`, `ExtractPatterns`, `FNV`, `Format`, `Hex`, `Int`, `IsBool`, `IsDouble`, `IsInt`, `IsList`, `IsMap`, `IsMatch`, `IsRootSpan`, `IsString`, `Len`, `Log`, `MD5`, `ParseJSON`, `ParseKeyValue`, `SHA1`, `SHA256`, `SHA512`, `Sort`, `SpanID`, `Split`, `String`, `Substring`, `TraceID`, `UUID`.

**Not simulated** (validated, but reported as *not simulated* when executed — `OTTL-SIM-009`): time and duration functions (`Time`, `Now`, `Duration`, `FormatTime`, `TruncateTime`, `Unix*`, `Hour`…`Weekday`, `Hours`…`Nanoseconds`), `URL`, `UserAgent`, `ParseCSV`, `ParseXML`, `ParseSimplifiedXML`, `Decode`, `Base64Decode`, `ExtractGrokPatterns`, `SliceToMap`, `ToKeyValueString`, `extract_sum_metric`, `extract_count_metric`, `copy_metric`, `aggregate_on_attributes`, `convert_summary_*`, routing connector `route()` execution, `trace_state` keys, `scale_metric` on histograms/summaries.

**Known approximations:** `limit` keeps keys in insertion order (Go map order is unspecified); `truncate_all` and `Substring` count characters (Go counts bytes, which differs for non-ASCII); `Concat` with `nil` and `ConvertCase(…, "camel")` edge cases.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `all` | `generate` (English → OTTL), `validate` (lint OTTL/config), `simulate` (run on telemetry), `all`. |
| `requirements` | string or array | — | Plain-English requests (max 20,000 characters). |
| `statements` | array | — | Your OTTL: strings, or `{"context": "span", "statements": [...]}` / `{"context": "log", "conditions": [...]}` objects (max 500). |
| `statementContext` | string | `auto` | Context for plain string statements; `auto` infers it from the paths. |
| `statementType` | string | `transform` | `transform` (editor statements) or `filter` (boolean conditions). |
| `collectorConfig` | string | — | Full Collector YAML (max 2 MiB): transform/filter processors and routing connector statements are extracted, linted and simulated in pipeline order. |
| `sampleTelemetry` | object | bundled | OTLP/JSON (`resourceSpans` / `resourceLogs` / `resourceMetrics`, camelCase or snake_case) or simplified `{"resource": {...}, "spans": [], "logs": [], "metrics": []}`. Up to 2,000 items per signal are simulated. |
| `requests` | array | — | Several requests (objects with the fields above plus `name`); one dataset item and charge each. |
| `collectorVersion` | string | `0.110.0` | Your Collector version; picks the syntax style and version checks. |
| `syntaxStyle` | string | `auto` | `auto`, `grouped` or `flat`. |
| `processorName` | string | `generated` | Generated processors are `transform/<name>` and `filter/<name>`. |
| `signals` | array | all | Signals that requirements apply to when they don't name one. |
| `errorMode` | string | `ignore` | `error_mode` of generated processors and default for simulation. |
| `useBundledSamples` | boolean | `true` | Simulate on bundled telemetry when `sampleTelemetry` is empty. |
| `dropWarningPercent` | integer | `50` | `OTTL-SIM-001` threshold. |
| `maxDiffItems` | integer | `25` | Maximum item diffs in the output (0–500). |
| `failOnSeverity` | string | `none` | CI gate: fail the run when a finding is at or above this severity. |
| `disabledRules` | array | — | Rule IDs to skip (custom-rules feature). |
| `licenseKey` | string (secret) | — | Enterprise / prepaid license. |

### Output

One dataset item per request, in the same `Report` format as every Michigan Public Technology Actor (shortened):

```json
{
  "product": "ottl-studio",
  "target": "request",
  "status": "ok",
  "score": 100,
  "summary": {"critical": 0, "high": 0, "medium": 0, "low": 0, "info": 4},
  "findings": [{"rule_id": "OTTL-SEM-001", "severity": "info",
                "location": "processors.transform/generated.log_statements[1].statements[0]", "…": "…"}],
  "data": {
    "mode": "all",
    "collector_version": "0.110.0",
    "syntax_style": {"style": "grouped", "reason": "classic `context:` groups work on every Collector release with OTTL"},
    "intents": [{"intent": "delete_attributes", "phrase": "Remove Kubernetes pod UIDs", "confidence": 0.85, "…": "…"}],
    "unrecognized": [],
    "generated": {
      "collector_yaml": "processors:\n  filter/generated:\n    error_mode: ignore\n    traces:\n      span:\n        - '(IsMatch(attributes[\"http.route\"], \"^/(health|healthz|…)/?$$\") …) and status.code != STATUS_CODE_ERROR'\n  transform/generated:\n    error_mode: ignore\n    trace_statements:\n      - context: resource\n        statements:\n          - 'delete_key(attributes, \"k8s.pod.uid\")'\n …",
      "pipeline_snippet": "service:\n  pipelines:\n    traces:\n      processors: [memory_limiter, filter/generated, transform/generated, batch]\n …",
      "statements": [{"signal": "traces", "context": "resource", "statement": "delete_key(attributes, \"k8s.pod.uid\")", "intent": "delete_attributes"}],
      "revalidation_findings": []
    },
    "simulation": {
      "telemetry_source": "bundled",
      "processor_order": {"traces": ["filter/generated", "transform/generated"]},
      "stats": {"signals": {"traces": {"items_in": 10, "items_out": 8, "items_dropped": 2, "bytes_reduction_pct": 16.3}},
                "attributes_removed": 21, "values_changed": 10, "bytes_reduction_pct": 10.9},
      "diffs": [{"item": "checkout · span 'GET /healthz'", "dropped": true},
                {"item": "checkout · span 'SELECT shop.orders'", "removed": ["resource.attributes[\"k8s.pod.uid\"]"],
                 "changed": {"attributes[\"db.statement\"]": {"before": "… 'jane.doe@example.com' …", "after": "… '[REDACTED_EMAIL]' …"}}}]
    },
    "warnings": [],
    "version_notes": {"disclaimer": "…", "notes": [{"version": "~0.120", "component": "transform", "note": "…"}]}
  }
}
```

`validate` requests add `data.validation` (statements with their inferred contexts, processors and pipelines from your config). The key-value store gets `OUTPUT` (run summary), `REPORT.md` and `REPORT.sarif`.

### Pricing

Pay per event — you only pay for successful work:

| Event | Price | When |
|---|---|---|
| `transform-generated` | **$0.10** per request | OTTL was generated from requirements and/or your statements / config were validated (batch run or any Standby `POST`). |
| `simulation-run` | **$0.05** per request | In addition, only when a simulation actually ran (`mode` `simulate` or `all`). |

Free: requests where nothing was recognised, unparsable YAML/JSON, simulate-only requests with unusable telemetry, invalid input, `GET /`, `/intents`, `/functions`, `/rules`. Example: 20 generate-and-simulate requests ≈ 20 × $0.15 = $3. Set a **maximum cost per run** in Apify: the Actor checks the remaining budget before each request and skips the simulation (with a warning) when only the generation is affordable, so you are never charged beyond it. Standard Apify platform usage applies.

### Limits and safety

- **Sizes:** requirements 20,000 characters, 500 statements, Collector config 2 MiB, Standby bodies 4 MB, 2,000 simulated items per signal per request, up to 100 `requests` per batch run on the Store plan.
- **No network, no LLM:** the Actor makes no outbound requests; everything runs from bundled catalogs.
- **Your data stays in your run:** telemetry and configs are processed in memory in your own Apify run (or the warm Standby instance serving your request); results go only to your dataset / key-value store or the HTTP response.
- **Generated configs are a starting point.** Review them and run `otelcol validate` with your real Collector build before deploying.

### Plans and licensing

Store users need nothing beyond an Apify account: pay-per-event pricing applies, and SARIF plus the CI gate are included. Disabling rules (`disabledRules`) and enterprise/prepaid options are described in [docs/licensing.md](../../docs/licensing.md). Local runs without a license use the evaluation plan (3 requests per run).

### Limitations

- **Intent recognition is pattern-based**, not an LLM: it covers the 61 intents above and their common phrasings. Complex or conditional requests ("only for service X on weekdays") may need hand-written OTTL — which you can then validate and simulate here.
- **The simulator models a subset of OTTL** (see *Simulation*); unsupported functions are reported, never faked. Time-based logic, grok, URL/User-Agent parsing and metric-creating functions are not executed.
- **Version knowledge is a bundled, best-effort table.** Function `since` versions are approximate; the grouped/flat syntax split was verified on the Collector builds listed above.
- **Lint locations** are YAML paths plus statement-relative line:column, not line numbers in your YAML file.
- **Routing connector** statements are extracted and linted but not executed.
- Per-signal pipeline wiring of a supplied config is checked for missing processors, not for every possible misconfiguration.

### FAQ

**Does it need my Collector or an LLM?** No. Everything is deterministic and offline; the same input always gives the same output.

**Which syntax should I use?** Leave `syntaxStyle` on `auto` and set `collectorVersion`. Flat statements are shorter and easier to read on 0.120+; grouped statements work everywhere.

**Why does my regex need `$$`?** The Collector expands `${env:VAR}` and `$VAR` in config files, so a literal `$` in a regex anchor must be written `$$`. Generated YAML already does this, and `OTTL-CFG-007` flags it in yours.

**Can I simulate my own telemetry?** Yes — paste OTLP/JSON (e.g. from the Collector `file` exporter or `OTEL_EXPORTER_OTLP_PROTOCOL=http/json`) or the simplified shape into `sampleTelemetry`.

**What if a requirement isn't understood?** It is listed under `data.unrecognized` with the closest intents and example phrasings. If nothing at all was recognised, the request is marked `failed` and is free.

**What happens when my spending limit is reached?** The Actor stops before the next paid request (or skips the simulation if only generation is affordable); results already produced are kept.

### Changelog

See [CHANGELOG.md](CHANGELOG.md).

# Changelog

This Actor's version history is a separate document: https://apify.com/michiganpublictech/ottl-studio/changelog.md

# Actor input Schema

## `mode` (type: `string`):

generate = plain English → OTTL; validate = lint your OTTL/config; simulate = run statements on sample telemetry; all = everything (default).

## `requirements` (type: `string`):

One or more transformation requests, e.g. 'Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans.' Unrecognised phrases are reported with the closest supported intents — nothing is guessed.

## `statements` (type: `array`):

Your OTTL to validate/simulate: a list of strings, or objects {"context": "span", "statements": \[...]} / {"context": "log", "conditions": \[...]}.

## `statementContext` (type: `string`):

Context for string statements; auto infers it from the paths.

## `statementType` (type: `string`):

transform = editor statements (set, delete_key…); filter = boolean conditions for the filter processor.

## `collectorConfig` (type: `string`):

A full Collector config; transform/filter processors and routing connector statements are extracted, linted and simulated in pipeline order.

## `sampleTelemetry` (type: `object`):

OTLP/JSON traces/logs/metrics (resourceSpans/resourceLogs/resourceMetrics) or the simplified {spans:\[], logs:\[], metrics:\[]} shape. Bundled realistic samples are used when empty.

## `requests` (type: `array`):

Optional: several requests, each an object with the fields above (requirements/statements/collectorConfig/sampleTelemetry/mode/name). One dataset item and charge per request.

## `collectorVersion` (type: `string`):

Your Collector version, e.g. 0.110.0. 0.120+ gets flat context-prefixed statements (best-effort version table); older gets classic `context:` groups.

## `syntaxStyle` (type: `string`):

auto picks from collectorVersion.

## `processorName` (type: `string`):

Name suffix for generated processors: transform/<name>, filter/<name>.

## `signals` (type: `array`):

Signals generated statements apply to when a requirement doesn't name one.

## `errorMode` (type: `string`):

ignore (recommended), silent, or propagate (one failing item rejects the whole batch).

## `useBundledSamples` (type: `boolean`):

Simulate against bundled representative telemetry when no sample telemetry is supplied.

## `dropWarningPercent` (type: `integer`):

Warn when filters drop at least this share of sample items.

## `maxDiffItems` (type: `integer`):

Maximum before/after item diffs in the output.

## `failOnSeverity` (type: `string`):

Fail the run (non-zero exit, useful in CI) when any finding is at or above this severity.

## `disabledRules` (type: `array`):

Rule IDs to skip, e.g. OTTL-SEM-001. Requires a plan with the custom rules feature.

## `licenseKey` (type: `string`):

Optional enterprise or prepaid license key. Leave empty to pay per event on Apify.

## Actor input object example

```json
{
  "mode": "all",
  "requirements": "Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans.",
  "statementContext": "auto",
  "statementType": "transform",
  "collectorVersion": "0.110.0",
  "syntaxStyle": "auto",
  "processorName": "generated",
  "signals": [
    "traces",
    "logs",
    "metrics"
  ],
  "errorMode": "ignore",
  "useBundledSamples": true,
  "dropWarningPercent": 50,
  "maxDiffItems": 25,
  "failOnSeverity": "none"
}
```

# Actor output Schema

## `results` (type: `string`):

One item per request: intents, generated OTTL + Collector YAML, validation findings, simulation diff and stats.

## `summary` (type: `string`):

Plan, per-request scores and billing summary (OUTPUT record).

## `reportMarkdown` (type: `string`):

Human-readable findings (REPORT.md).

## `reportSarif` (type: `string`):

SARIF 2.1.0 findings for CI (REPORT.sarif, when entitled).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "requirements": "Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans."
};

// Run the Actor and wait for it to finish
const run = await client.actor("michiganpublictech/ottl-studio").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "requirements": "Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans." }

# Run the Actor and wait for it to finish
run = client.actor("michiganpublictech/ottl-studio").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "requirements": "Remove Kubernetes pod UIDs, redact user email addresses, and drop successful health-check spans."
}' |
apify call michiganpublictech/ottl-studio --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,michiganpublictech/ottl-studio"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iw9ybg9kgOFV675M1/builds/HM1927Rgk7DheypbD/openapi.json