# Changelog of OTel PII & Secrets Leak Detector (`michiganpublictech/otel-pii-leak-detector`) Actor

- **URL**: https://apify.com/michiganpublictech/otel-pii-leak-detector/changelog.md
- **Full Actor documentation**: https://apify.com/michiganpublictech/otel-pii-leak-detector.md

## Changelog — OTel PII & Secrets Leak Detector

### 0.1.0 — unreleased

- Initial version.
- Input: OTLP/JSON traces, logs and metrics (auto-detected; snake_case and legacy `instrumentationLibrary*` keys
  accepted), a simplified `{resource, scope, spans, logs, metrics}` shape, `telemetryBatches` and JSON Lines text.
- Scans resource/scope attributes, span names, attributes, events, links and status messages, log bodies
  (recursively through structured `kvlistValue`/`arrayValue` bodies) and attributes, metric names/descriptions,
  data-point attributes and exemplar filtered attributes; string, int and double values.
- 66 rules (`LEAK-001`…`LEAK-066`) in nine detector groups: secrets, credentials, PII contact, PII identifiers,
  network, financial, health, location and free text (off by default), plus custom patterns and two placement rules
  (`LEAK-057`, `LEAK-058`). Checksum validation (Luhn, IBAN, ABA, DNI/NIE, BSN, IMEI, CPF, Aadhaar Verhoeff,
  PESEL), SSA/NANP rules, structural JWT decoding (PII claims, never verified), known test values downgraded,
  placeholders and UUIDs/trace IDs ignored.
- Detectors added after the first internal build: Brazilian CPF (`LEAK-059`), Indian Aadhaar (`LEAK-060`), Polish
  PESEL (`LEAK-061`), Shopify tokens (`LEAK-062`), HashiCorp Vault / Terraform Cloud tokens (`LEAK-063`), Hugging
  Face tokens (`LEAK-064`), Databricks PATs (`LEAK-065`) and Mailgun keys with a Mailgun label (`LEAK-066`).
- Precision tuning from a synthetic-corpus regression test (~200k benign values, 603 planted leaks): pagination /
  continuation tokens are not reported as secrets; `key=`/`code=`/`sig=` query parameters need a secret-looking
  value; `password: too_short`-style status words are ignored; k8s pod conditions and experiment treatments are not
  health data; UK NINO needs a label or the printed layout; unlabelled DNI-shaped and card-shaped digit runs are
  low confidence, or skipped when they are the whole value of an ID-like attribute; public IPs followed by a server
  port or after "connect to"/"upstream" are not reported; 12-digit grouped card numbers are medium confidence.
  Measured result: recall 1.0, no high-confidence false positives, benign medium-confidence FP rate ≈ 0.013 %.
- Risk report: per-category counts, services, scopes, top leaking attribute keys, confidence, and potential
  GDPR/CCPA/HIPAA/PCI DSS exposure labels (explicitly not legal advice).
- Remediation: generated OpenTelemetry Collector config (`attributes` delete, `transform` OTTL salted SHA256
  hashing and pattern replacement, `redaction` safety net) wired between `memory_limiter` and `batch`, with
  `contrib` / `contrib-legacy` flavors, SDK-side hints and a Python emulation that reports what the config would
  miss; sanitized telemetry with a re-scan self-check.
- Collector config verified against real otelcol-contrib 0.115.1, 0.148.0 and 0.150.1 (`validate`, plus end-to-end
  runs whose output matched the emulation). Fixes from that: `contrib-legacy` now wires `redaction` into logs and
  metrics too (0.115.1 supports it) and only drops `blocked_key_patterns` (rejected by 0.115.1); `contrib` uses the
  `otlp_grpc` exporter (`otlp` is deprecated in 0.150.1); hashed keys go into `ignored_keys` so a blocked key
  pattern doesn't overwrite the hash with `****`; the `drop` strategy's `allowed_keys` now covers resource/scope
  attributes, map attributes and log-body keys and always keeps `service.name` (before, the redaction processor
  stripped every resource attribute); the emulation models where each version's redaction processor acts. Frozen
  snapshot configs are tested offline.
- Privacy: raw values never stored or logged — type-aware redacted previews (passwords always `****`) and salted
  HMAC fingerprints; `sampleRetention: none` drops both.
- ReDoS guard for `customPatterns` / `allowlistPatterns` (nested quantifiers, overlapping alternations and
  backreferences rejected; scanned values capped at 64 KiB).
- Actor Standby HTTP API: `POST /scan`, `GET /rules`, `GET /detectors`, readiness probe.
- Pay-per-event: `telemetry-scanned` ($0.25 per batch unit of ≤ 5 MB / ≤ 10,000 records) and `sanitized-output`
  ($0.10 when a remediation pack with at least one change is delivered). Failed or refused batches are free.
- Roadmap / known caveats: see [STATUS.md](STATUS.md).