OTel Cost & Cardinality Analyzer

Describe your OpenTelemetry metrics, or paste an OTLP/JSON sample or Prometheus /metrics text, to see how many active series and samples they produce, what that costs each month, which attributes drive the cost, and how much each fix saves. You also get an optimized Collector configuration that applies the fixes.

Metric cost is driven by cardinality. Every distinct combination of attribute values becomes its own time series, and histograms multiply that by their bucket count. A single user.id on http.server.request.duration can turn a few hundred series into hundreds of thousands. Other common culprits are an un-templated url.path, an exact status code, a 5-second export interval and a legacy metric still emitted during a semantic-convention migration. These cost money every month, and no SDK or Collector will warn you about them.

This Actor estimates series, samples and monthly cost per metric. It ranks the attributes whose removal saves the most and runs 47 rules (CARD-*) for unbounded identifiers, redundant metrics, histogram buckets, export intervals, temporality and resource attributes. Every recommendation comes with quantified savings: series, samples per month and money. The Collector-applicable fixes are turned into valid Collector YAML: filter, OTTL transform with delete_key(attributes, "…"), resource and metricstransform processors. If you supply your existing configuration, the processors are merged into it.

Estimates, not invoices. Worst-case series are upper bounds: every attribute combination occurring at once. Realistic figures are heuristics unless you supply observed data. Preset prices are illustrative assumptions, not vendor quotes. Enter your own contract prices for real numbers.

---

Why use it

• Find the expensive attributes before the bill does. The Actor ranks every attribute by the series and money you would save by removing it. This covers data-point and resource attributes, per metric.
• Catch unbounded identifiers. It checks attribute names (user.id, session.id, request.id, trace_id, client.address, url.full, user.email, created_at, db.statement, …). It also checks the shape of sample values: UUIDs, hashes, numeric IDs, IPs, e-mails, timestamps, URLs and raw paths such as /users/8412/orders.
• Remove redundancy. It flags legacy and stable semantic-convention metrics emitted side by side (http.server.duration and http.server.request.duration). It also flags counters that duplicate a histogram's _count, metrics defined twice, and legacy/stable attribute pairs (http.method and http.request.method).
• Right-size histograms and intervals. It flags explicit histograms with more than 20 buckets, oversized exponential histograms, seconds-unit histograms that use millisecond buckets, and export intervals under 10 or 30 seconds.
• Get a fix, not only a report. You get an optimized Collector configuration that is delta-aware: what your Collector already drops is taken into account and never recommended again. You also get ready-to-paste SDK Views for Python, Java, Go, .NET and JavaScript, plus hints (OTEL_METRIC_EXPORT_INTERVAL, exponential histograms, deltatocumulative / cumulativetodelta) for the changes the Collector can't make safely.

Who it's for

• Platform / observability teams who own the metrics bill and need to prioritise fixes across services.
• FinOps and engineering managers who want a per-metric cost breakdown and a savings estimate before committing engineering time.
• Developers adding instrumentation. Run it in CI and fail the pull request that adds user.id to a histogram.
• Consultancies who audit customer telemetry and need a repeatable, explainable report.

Features

Area	What you get
Estimation	Worst-case series (upper bound), realistic series, samples per month, monthly cost before / after / worst case, per metric and in total
Inputs	Metric definitions with distinct-value counts or sample values, OTLP/JSON samples and Prometheus text exposition (cardinality counted from data points), an existing Collector YAML
Contributors	Top 10 attributes by series and cost saved if removed
Rules	47 rules: identifiers, value shapes, raw routes, status codes, series levels, budgets, histograms, redundancy, temporality, export interval, resource attributes, spanmetrics connector, pricing
Recommendations	Drop an attribute, aggregate one (route templating, status code → class), drop a metric, reduce buckets, lower the exponential max_size, increase the interval. Each comes with series, samples and cost saved
Optimized config	filter/cardinality, resource/cardinality, transform/cardinality (OTTL), metricstransform/cardinality (aggregate_labels), merged and wired after memory_limiter and before batch. Output shapes are validated against otelcol-contrib 0.115 and 0.150
SDK Views	One View per instrument (attribute allow-list, explicit buckets, exponential max_size, drop) rendered for Python, Java, Go, .NET and JavaScript
Reports	Dataset item per analysis, REPORT.md, REPORT.sarif (GitHub code scanning), OUTPUT run summary
Batch	Analyse many services or environments in one run with analyses

How to use

In Apify Console

1. Open the Actor and keep the prefilled example, or replace Metric definitions with your own metrics.
2. Optionally paste an OTLP/JSON metrics sample (for example from the Collector file exporter), Prometheus /metrics text, and your existing Collector configuration.
3. Set your prices (or keep the illustrative preset) and optional budgets.
4. Click Start. Open the Output tab to see the views: Overview, Recommendations, Findings, Per-metric estimates and Optimized configuration.

Via the API (curl)

curl -s -X POST \
  "https://api.apify.com/v2/acts/michiganpublictech~otel-cardinality-analyzer/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d @- <<'EOF'
{
  "analysisName": "checkout-service",
  "metrics": [
    {"name": "http.server.request.duration", "type": "histogram", "unit": "s",
     "attributes": {"http.request.method": 6, "http.route": 35, "http.response.status_code": 14, "user.id": 5000}},
    {"name": "orders.created", "type": "counter", "frequency_seconds": 10,
     "attributes": {"payment.method": 5, "order.id": 50000}}
  ],
  "resourceAttributes": {"service.name": 1, "k8s.pod.name": 12},
  "costPerActiveSeriesPerMonth": 0.008,
  "currency": "USD"
}
EOF

To analyse a real OTLP sample, write it with the Collector file exporter (format: json, one export per line) and merge the exports into one request. otlpMetrics must be a JSON object: Apify rejects a list or a string there before the run starts. Several exports of the same streams let the Actor infer the export interval.

jq -s '{otlpMetrics: {resourceMetrics: [.[].resourceMetrics[]]}, costPerMillionSamples: 0.30}' metrics.jsonl |
  curl -s -X POST -H "Content-Type: application/json" -d @- \
  "https://api.apify.com/v2/acts/michiganpublictech~otel-cardinality-analyzer/run-sync-get-dataset-items?token=$APIFY_TOKEN" |
  jq '.[0].data | {estimated_series_realistic, est_monthly_cost_before, est_monthly_cost_after, top: .top_contributors[0:3]}'

Python (apify-client)

from pathlib import Path
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("michiganpublictech/otel-cardinality-analyzer").call(
    run_input={
        "metrics": [...],  # your metric definitions
        "collectorConfig": Path("otel-collector.yaml").read_text(),
        "costPerActiveSeriesPerMonth": 0.008,
    }
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    d = item["data"]
    print(
        item["target"], d["estimated_series_realistic"], d["est_monthly_cost_before"], "->", d["est_monthly_cost_after"]
    )
    for rec in d["recommendations"][:5]:
        print(
            f"  {rec['id']} {rec['title']}: saves {rec['savings']['monthly_cost']} {rec['savings']['currency']}/month"
        )
    if d["optimized_config"]:
        Path("otel-collector.optimized.yaml").write_text(d["optimized_config"])

CI: block cardinality regressions on pull requests

Keep a metrics.json (your metric inventory, or a captured OTLP sample) in the repository. Fail the build when a change introduces a critical finding, such as a per-request ID on a metric:

name: metrics-cardinality
on: [pull_request]
jobs:
  cardinality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Analyse metric cardinality
        env:
          APIFY_TOKEN: ${{ secrets.APIFY_TOKEN }}
        run: |
          jq '{metrics: ., failOnSeverity: "critical", costPerActiveSeriesPerMonth: 0.008}' metrics.json > input.json
          curl -sf -X POST -H "Content-Type: application/json" -d @input.json \
            "https://api.apify.com/v2/acts/michiganpublictech~otel-cardinality-analyzer/runs?token=$APIFY_TOKEN&waitForFinish=300" \
            | jq -e '.data.status == "SUCCEEDED"'

Input

Everything at the top level forms one analysis. Add more in analyses (batch mode). Each analysis is billed once.

Field	Type	Description
metrics	array	Metric definitions (see below).
otlpMetrics	object	One OTLP/JSON ExportMetricsServiceRequest. To combine several exports, merge their resourceMetrics arrays into one object. Cardinality is counted from the data points. (Inside analyses items, a list of requests or a JSON Lines string also works.)
prometheusMetrics	string	Prometheus text exposition (/metrics output or OpenMetrics text). Families come from # TYPE lines (or are inferred from le / quantile); histogram boundaries and summary quantiles are read; job and instance count as resource attributes. Set defaultExportIntervalSeconds to your scrape interval, or concatenate timestamped scrapes to infer it.
analysisName	string	Label for the analysis.
resourceAttributes	object	Resource attributes applied to every metric in metrics, for example {"service.name": 1, "k8s.pod.name": 20}.
collectorConfig	string	Your existing Collector YAML. It makes recommendations delta-aware, and the optimized config is merged into it.
analyses	array	Batch mode: [{name, metrics, otlpMetrics, prometheusMetrics, collectorConfig?, resourceAttributes?, …}]. Missing settings are inherited from the top level.
vendorPreset	enum	generic (illustrative 0.005 per active series per month), generic-samples (illustrative 0.25 per million samples), custom (only your prices).
costPerActiveSeriesPerMonth	number	Your price per active series per month. It overrides the preset.
costPerMillionSamples	number	Your price per million samples (data points). It overrides the preset; you can set both.
currency	string	Label for cost figures (default USD; no conversion).
monthlySeriesBudget / monthlyCostBudget	number	Optional budgets. Exceeding one is a finding.
defaultExportIntervalSeconds	integer	Interval for metrics that don't set one (default 60, the SDK default).
combinationFactor	number 0–1	Fraction of worst-case combinations that really occur. Leave empty to use the heuristic.
exponentialHistogramMode	enum	expanded (each populated bucket is a series, the conservative choice) or native.
backendTemporality	enum	any, cumulative (Prometheus-compatible) or delta. Used for the temporality checks.
optimize	boolean	Generate the optimized Collector configuration (default true).
minSeverity	enum	Hide findings below this severity. Estimates and score are unaffected.
failOnSeverity	enum	CI gate: fail the run at or above this severity.
disabledRules	array	Rule IDs to skip. This also removes their recommendations.
licenseKey	secret	Optional enterprise or prepaid key.

Metric definition fields (snake_case or camelCase both accepted):

{
  "name": "http.server.request.duration",
  "type": "histogram",
  "unit": "s",
  "temporality": "cumulative",
  "frequency_seconds": 60,
  "bucket_boundaries": [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
  "attributes": {
    "http.request.method": 6,
    "http.route": ["/users/{id}", "/orders/{id}", "/health"],
    "url.path": {"cardinality": 2000, "samples": ["/v1/charges/8812", "/v1/charges/9921/refunds"]}
  },
  "resource_attributes": {"k8s.pod.name": 12},
  "combination_factor": 0.05,
  "estimated_series": 1800
}

• type: counter, updowncounter, gauge, histogram, exponential_histogram or summary. Aliases such as sum and observable_gauge are accepted. If you leave it out, the type is inferred from bucket fields, or the metric is treated as a gauge.
• attributes: a distinct-value count, a list of sample values (counted, and checked for ID-like shapes) or {cardinality, samples}. For active-series billing, count the values that are active at the same time (for example the replica count, not every pod seen during the month).
• Histograms: histogram_buckets (bucket count, including +Inf) or bucket_boundaries. The default is the SDK's 16 buckets. Exponential histograms: exp_max_size (default 160).
• estimated_series is your own measured number. It replaces the heuristic and is rescaled when a recommendation removes an attribute.

Output

One dataset item per analysis, in the shared report format used by all our Actors (target, score, risk_level, summary, findings[], errors[], disclaimer). The product data is under data:

{
  "target": "checkout-service (production)",
  "score": 5,
  "risk_level": "critical",
  "data": {
    "estimated_series_worst_case": 3007619040,
    "estimated_series_realistic": 890728,
    "samples_per_month": 68537577600,
    "currency": "USD",
    "est_monthly_cost_before": 4453.64,
    "est_monthly_cost_worst_case": 15038095.2,
    "est_monthly_cost_after": 140.91,
    "est_monthly_savings": 4312.73,
    "savings_percent": 96.8,
    "estimated_series_after": 28182,
    "cardinality_risk": "critical",
    "pricing": {"preset": "generic", "cost_per_active_series_per_month": 0.005, "illustrative_assumption": true, "...": "..."},
    "top_contributors": [
      {"metric": "http.server.request.duration", "attribute": "user.id", "distinct_values": 5000,
       "series_saved_if_removed": 556781, "monthly_cost_saved_if_removed": 2783.91, "share_of_total_series": 0.6251}
    ],
    "recommendations": [
      {"id": "REC-001", "action": "drop_attribute", "title": "Drop attribute `user.id` from `http.server.request.duration`",
       "rule_id": "CARD-ID-001", "where": "collector", "in_optimized_config": true,
       "savings": {"series": 556781, "samples_per_month": 24052939200, "monthly_cost": 2783.91, "percent_of_cost": 62.5}}
    ],
    "metrics": [{"name": "orders.created", "estimated_series_worst_case": 3000000, "estimated_series_realistic": 139158, "risk": "critical", "...": "..."}],
    "metrics_after": ["..."],
    "optimized_config": "# Optimized by OTel Cost & Cardinality Analyzer v0.1.0 ...",
    "optimized_config_changes": [{"path": "processors.transform/cardinality", "description": "OTTL: delete 2 attribute(s), ..."}],
    "hints": [{"kind": "export-interval", "title": "Increase the SDK export interval to 60s", "snippet": "..."}],
    "existing_collector": {"metrics_pipelines": ["metrics"], "dropped_attributes": {}, "...": "..."},
    "assumptions": ["Worst-case series are upper bounds: ...", "Prices are the 'generic' preset — ILLUSTRATIVE ASSUMPTIONS ..."],
    "notes": []
  }
}

• Per-recommendation savings are measured against the baseline, one recommendation at a time. est_monthly_cost_after applies all recommendations together, so the per-recommendation savings don't add up to the total when they overlap.
• cardinality_risk: per metric, critical means ≥ 100,000 realistic series, high ≥ 10,000 and medium ≥ 1,000. Per analysis, critical means ≥ 1,000,000, high ≥ 100,000 and medium ≥ 10,000. The worse of the two is reported.
• Key-value store: OUTPUT (run summary), REPORT.md (human-readable) and REPORT.sarif (GitHub code scanning, on plans with SARIF).

The rule catalog with descriptions and references is in RULES.md.

How the estimates work

• Worst case (upper bound) = product of every data-point attribute's distinct values × product of every resource attribute's distinct values × series per attribute set. Series per attribute set is: histogram buckets + 2 (_sum, _count); populated buckets + 2 for expanded exponential histograms, or 1 when stored natively; quantiles + 2 for summaries; 1 for counters and gauges. It assumes every combination occurs, which real traffic almost never does.
• Realistic, in order of preference:
  1. Observed distinct streams in your OTLP sample.
  2. Your estimated_series.
  3. Worst case × combinationFactor.
  4. A correlation-aware heuristic. Correlated attributes count once: pod UID / pod name / container ID, and legacy and stable names of the same attribute. The largest remaining factor counts fully and the others with halving exponents, because real combinations are sparse. Replicas multiply fully, and are dampened when a per-entity identifier dominates.
• Samples per month = series × 2,592,000 s (30 days) / export interval.
• Cost = series × price per active series + samples / 1,000,000 × price per million samples.
• Delta-aware: if you provide collectorConfig, processors that are wired into a metrics pipeline are applied before estimating: filter, transform (delete_key, delete_matching_keys, keep_keys, replace_pattern, aggregate_on_attributes), attributes, resource, metricstransform aggregate_labels and interval. The before-Collector totals are reported in data.before_existing_collector.
• Only unconditional actions count. An existing action is credited only for the metrics it always applies to. The Actor reads OTTL where clauses and block-level conditions (metric.name == / !=, or, and, parentheses, IsMatch, HasPrefix, HasSuffix), attributes processor include / exclude metric_names (strict or regexp), metricstransform regexp include, and filter conditions joined with or. A condition on anything else, such as attributes["env"] == "prod", metric.type or resources, applies to only some data points. Those actions are listed in data.existing_collector.conditional_not_credited and the fix is still recommended.

The optimized Collector configuration

Processor	What it does
filter/cardinality	Drops redundant metrics (name == "http.server.duration").
resource/cardinality	Deletes exploding resource attributes (for example host.name on serverless, redundant k8s.pod.uid).
transform/cardinality	OTTL per metric: delete_key(attributes, "user.id") where metric.name == "…", route templating with replace_pattern(attributes["url.path"], "/[0-9]+", "/{id}"), and status code → http.response.status_class.
metricstransform/cardinality	aggregate_labels re-aggregates points that became identical after attributes were removed: sum for counters and histograms, max for gauges. Without it, backends see duplicate streams.

The processors are inserted into every metrics pipeline after memory_limiter and before batch. Without a collectorConfig, the Actor generates a minimal, complete config (OTLP receiver, memory_limiter, the new processors, batch, otlphttp exporter with ${env:OTLP_EXPORTER_ENDPOINT:-http://localhost:4318}). Hard-coded credentials in a supplied config are replaced by ${env:…} placeholders in the output.

Validated against real Collectors. The generated shapes pass otelcol-contrib validate on 0.115.1, 0.148.0 and 0.150.1. These include every processor type, aggregate_labels with sum and max, ID collisions (transform/cardinality_2), several metrics pipelines and names that need escaping. The fixture configuration was also run end to end on 0.115.1 and 0.150.1, and the drops, route templating, status classes (including string codes) and re-aggregation behaved as intended. The filter uses bare name == "…" because 0.115 rejects metric.name in the filter's metric context. Still run otelcol validate --config=<file> against your own distribution before deploying.

Some changes can't be made safely in the Collector, such as bucket boundaries, the export interval and exponential max_size. Those are returned as hints (not charged):

• sdk-views: one View per instrument that combines every change for it: attribute allow-list, explicit bucket boundaries (scaled for ms), base-2 exponential max_size, or drop. Two Views matching one instrument would export it twice, so the changes are combined. The hint includes code for Python, Java, Go, .NET and JavaScript (snippets.<language>) and the structured views list. The Python, JavaScript (@opentelemetry/sdk-metrics 2.x) and Go output was executed against those SDKs. The Java and .NET output follows their documented View APIs.
• Environment-variable and optional-processor snippets: OTEL_METRIC_EXPORT_INTERVAL, interval, deltatocumulative, cumulativetodelta and exponential histograms for backends that store them natively.

Pricing

This Actor uses pay-per-event pricing. You pay only for results, and Apify platform usage is included.

Event	Price	When it is charged
analysis-completed	$0.25	Once per analysis: the top-level metrics count as one analysis, and each analyses entry is another
optimized-config	$0.10	Once per analysis that produced an optimized Collector configuration with at least one change

• Analyses that can't be parsed (malformed metrics, invalid OTLP) are free. They're still listed with the reason.
• Hints, findings without Collector changes, optimize: false and plans without optimized output never charge optimized-config.
• Apify's maximum cost per run is respected. The remaining budget is checked before every analysis. If it covers the analysis but not the optimized configuration, the analysis is published without the config and that event isn't charged.

Example: a batch of 8 services, 6 of which get an optimized configuration, costs 8 × $0.25 + 6 × $0.10 = $2.60.

Plans and licensing

Apify Store users get the full product on pay-per-event pricing: all rules, the optimized configuration, SARIF export, the CI gate and up to 100 analyses per run. License keys (licenseKey) unlock more. Pro adds disabledRules and up to 500 analyses per run. Team raises that to 2,000, and Enterprise adds white-label reports. Prepaid keys skip pay-per-event charges. Local runs without a key use the Community plan: no optimized configuration and at most 3 analyses per run. Missing features never fail a run; they're skipped with a note. See ../../docs/licensing.md.

Limitations

• Estimates, not measurements. The Actor doesn't connect to your backend or read your bill. Worst-case numbers are upper bounds. Realistic numbers are heuristics unless you provide an OTLP or Prometheus sample or estimated_series. An OTLP sample only shows the combinations present in that sample, so longer samples give better numbers.
• Prices are yours to set. Presets are generic, illustrative assumptions, not any vendor's list price. Vendors also bill differently: per series, per sample, per host, with tiers and commitments. Use the two price fields to model yours.
• Collector-side removal has limits. aggregate_labels merges points only within a batch. For cumulative metrics from many instances, an SDK View (attribute allow-list) is the most correct fix. Exponential histograms and summaries can't be re-aggregated by aggregate_labels, so you get an SDK hint instead.
• Heuristics can misfire. Identifier detection uses names and value shapes. Use disabledRules or minSeverity to tune noise.
• The optimized YAML loses comments and anchors. Review it and validate it with otelcol validate before deploying. OTTL functions (replace_pattern, Int) and metricstransform require the contrib distribution.
• Prometheus input uses Prometheus names. The Collector processors match metric names as the Collector's prometheus receiver sees them. For services that export OTLP, the OTel instrument names apply instead.

FAQ

Why are the worst-case numbers so big? They multiply every attribute's distinct values, as if every user hit every route on every pod with every status code. That is the theoretical ceiling, and it shows which attributes can explode. The realistic figure is what we expect in practice. Provide an OTLP sample to replace it with observed counts.

Are these Datadog / Grafana / New Relic prices? No. We don't ship vendor price lists, because they change and depend on your contract. The presets are clearly labelled illustrative assumptions. Enter your own costPerActiveSeriesPerMonth and/or costPerMillionSamples.

Does my data leave Apify? Your input is processed inside the Actor run. Sample attribute values are used only for counting and shape detection; findings report the detected shape, never the values. Hard-coded credentials in a supplied Collector config are replaced with ${env:…} placeholders in the output.

My Collector already drops user.id — will it still be recommended? No. Supply collectorConfig; anything a metrics-pipeline processor already removes is excluded, reported under CARD-CFG-003, and the before-Collector numbers are shown separately.

Should I remove attributes in the Collector or in the SDK? In the SDK when you can: Views aggregate before export, so they're the cheapest and most correct. The Collector configuration is the fastest way to stop the bleeding centrally, and it's what this Actor generates. The sdk-views hint gives you the matching View code for Python, Java, Go, .NET and JavaScript.

How do I silence a rule? Use disabledRules (plans with custom rules) or raise minSeverity. Disabling a rule also removes its recommendation. Rule IDs are stable and never reused.

Changelog

See CHANGELOG.md.