RAG Pipeline Latency Profiler

Stage-by-stage performance, tail latency, bottleneck ranking, SLO breaches, and instrumentation gaps for Retrieval-Augmented Generation pipelines.

This Actor turns RAG trace rows into a compact performance audit. It supports arbitrary pipeline stages and calculates p50, p90, p95, and p99 latency, error rate, observed latency share, end-to-end request latency when request IDs are available, and optimization priorities for the slowest stages.

Use cases

• production RAG observability
• weekly latency regression checks
• vector retrieval and reranking tuning
• LLM gateway and model-routing optimization
• AI platform SRE and FinOps teams
• pre-release performance baselines
• comparing pipeline changes before and after deployment

Why use this Actor

Key features

• p50, p90, p95, and p99 latency by arbitrary pipeline stage
• Request-level end-to-end timing when request IDs are available
• Stage and end-to-end SLO breach analysis
• Bottleneck ranking, instrumentation gaps, and explainable optimization priorities

An average latency number hides the problem engineers usually care about: tail behavior and where the time actually goes. This Actor reports:

• average, min, max, p50, p90, p95, and p99 by stage
• stage error rate
• share of observed span time by stage
• per-stage SLO breach rate when thresholds are supplied
• measured end-to-end request percentiles when request_id is present
• a ranked bottleneck list
• stage-specific optimization suggestions
• missing instrumentation and low-sample warnings

No external model, paid API, vector database, or browser is required. The Actor analyzes telemetry supplied in the input, keeping compute and platform cost low.

Input

rag_logs is required. Each row can be a JSON object or a JSON string. Every valid row needs at least:

• stage
• latency_ms

For stronger analysis, add request_id, status, or success.

Example

{
  "rag_logs": [
    {"request_id": "req-1", "stage": "embedding", "latency_ms": 82, "status": "ok"},
    {"request_id": "req-1", "stage": "retrieval", "latency_ms": 238, "status": "ok"},
    {"request_id": "req-1", "stage": "reranking", "latency_ms": 144, "status": "ok"},
    {"request_id": "req-1", "stage": "generation", "latency_ms": 1180, "status": "ok"}
  ],
  "expected_stages": ["embedding", "retrieval", "reranking", "generation"],
  "end_to_end_sla_ms": 2200,
  "stage_sla_ms": {
    "retrieval": 350,
    "generation": 1500
  }
}

Arbitrary stages

The profiler is not limited to the classic retrieval/embedding/generation trio. You can instrument stages such as:

• query rewriting
• cache lookup
• hybrid search
• graph traversal
• metadata filtering
• reranking
• prompt assembly
• guardrails
• model generation
• citation validation

Stage names are preserved exactly in the output.

End-to-end measurement

When spans include request_id, the Actor sums stage latency per request and reports measured request-level p50/p90/p95/p99 and SLO breach rate.

When request IDs are absent, the Actor reports only a clearly labelled additive estimate based on stage averages. It does not pretend that an inferred average is a measured end-to-end percentile.

Output

A run emits one default-dataset item containing the complete audit:

• stage_profiles
• bottleneck_ranking
• end_to_end
• recommendations
• instrumentation_gaps
• evidence confidence
• method note explaining measured versus estimated metrics

The same result is written to the OUTPUT key-value-store record.

Limitations

Responsible interpretation

Tail percentiles are unstable with very small samples. The Actor flags low-sample stages and reports confidence rather than dressing four traces in a lab coat and calling them production science.

Optimization suggestions are hypotheses to benchmark, not claims that a specific infrastructure change will always improve performance.

Data provenance and privacy

All calculations use only the trace rows you provide. The Actor does not send your telemetry to external model providers. Avoid including prompts, retrieved documents, secrets, or personal data when stage metadata and timings are sufficient.

Pricing

Pricing is pay per completed audit: one successfully emitted performance audit equals one billable default-dataset item. The Store price is activated only after the factory checks real Apify validation-run cost and margin.

Search terms

RAG latency profiler, RAG observability, RAG bottleneck analyzer, RAG performance benchmark, retrieval latency, LLM latency audit, RAG p95 p99, vector search performance, AI pipeline SLO, RAG monitoring.

Commercial intelligence layer

Gen2 adds baseline latency regression, projected request volume, optional infrastructure-cost impact, confidence scoring and a GO / WARN / BLOCK release decision on top of the existing stage percentiles, bottleneck ranking and SLO evidence.