RAG Pipeline Latency & SLO Bottleneck Profiler

Deterministic RAG performance audit for stage latency, tail percentiles, SLO breaches and bottleneck ranking. Gen2 adds baseline comparison, economic impact, confidence, and explicit automation decisions. Returns structured, machine-readable results for repeatable research.

Quick start

Start with this working example and replace the target values with your own:

{
  "rag_logs": [
    {
      "request_id": "r1",
      "stage": "embedding",
      "latency_ms": 80,
      "status": "ok"
    },
    {
      "request_id": "r1",
      "stage": "retrieval",
      "latency_ms": 240,
      "status": "ok"
    },
    {
      "request_id": "r1",
      "stage": "reranking",
      "latency_ms": 150,
      "status": "ok"
    },
    {
      "request_id": "r1",
      "stage": "generation",
      "latency_ms": 1100,
      "status": "ok"
    }
  ],
  "expected_stages": [
    "embedding",
    "retrieval",
    "reranking",
    "generation"
  ],
  "end_to_end_sla_ms": 2200,
  "stage_sla_ms": {
    "retrieval": 350,
    "generation": 1500
  }
}

The Actor writes structured results to the default Apify Dataset and can be used from the Store, API, schedules, Tasks, automations and MCP-compatible AI workflows.

Input

• rag_logs — RAG latency spans: Each row can be an object or JSON string with at least stage and latency_ms.
• expected_stages — Expected pipeline stages: Used to flag missing instrumentation. Arbitrary stage names remain supported.
• end_to_end_sla_ms — End-to-end SLO in milliseconds: Optional latency target. Measured breach rate requires request_id on spans.
• stage_sla_ms — Per-stage SLOs: Optional mapping from stage name to latency target in milliseconds.
• previous_profile — Previous latency profile: Optional prior Actor output for release-regression comparison.
• monthly_requests — Monthly RAG requests: Projected monthly request volume for economic impact estimates.

All integration and advanced analysis fields are optional. The default example is intentionally runnable without configuring MCP or a previous-run baseline.

Output

The Dataset exposes predictable machine-readable output. Representative fields include results.

Pricing

This Actor uses Pay Per Event. The current factory base price is $0.020000 per result event. The Apify Store remains the source of truth for the price and plan/tier details shown to the buyer.

Use cases

• Run the buyer-ready workflows exposed as Apify Tasks without preparing a custom integration first.
• Use the Actor from API or schedules for recurring collection, comparison or monitoring.
• Feed the structured Dataset output into spreadsheets, databases, automations or AI agents.
• Compare repeat runs when the product supports snapshots or previous-run inputs.

Automation and AI

Use the same Actor through Apify API, schedules, public Tasks and the Apify MCP server. Outputs are structured for downstream workflows and AI agents rather than requiring manual copy/paste.

Limitations

• Public websites and upstream APIs can change markup, access rules, rate limits or field availability without notice.
• Fields that are not publicly available are returned as unavailable or omitted rather than fabricated.
• Analytical outputs depend on the quality and coverage of the supplied or collected source data.
• Treat marketplace, reputation, workforce, safety or commercial signals as decision support and validate material decisions against the underlying source evidence.

Detailed documentation

Detailed documentation

Stage-by-stage performance, tail latency, bottleneck ranking, SLO breaches, and instrumentation gaps for Retrieval-Augmented Generation pipelines.

This Actor turns RAG trace rows into a compact performance audit. It supports arbitrary pipeline stages and calculates p50, p90, p95, and p99 latency, error rate, observed latency share, end-to-end request latency when request IDs are available, and optimization priorities for the slowest stages.

Use cases

• production RAG observability
• weekly latency regression checks
• vector retrieval and reranking tuning
• LLM gateway and model-routing optimization
• AI platform SRE and FinOps teams
• pre-release performance baselines
• comparing pipeline changes before and after deployment

Why use this Actor

Key features

• p50, p90, p95, and p99 latency by arbitrary pipeline stage
• Request-level end-to-end timing when request IDs are available
• Stage and end-to-end SLO breach analysis
• Bottleneck ranking, instrumentation gaps, and explainable optimization priorities

An average latency number hides the problem engineers usually care about: tail behavior and where the time actually goes. This Actor reports:

• average, min, max, p50, p90, p95, and p99 by stage
• stage error rate
• share of observed span time by stage
• per-stage SLO breach rate when thresholds are supplied
• measured end-to-end request percentiles when request_id is present
• a ranked bottleneck list
• stage-specific optimization suggestions
• missing instrumentation and low-sample warnings

No external model, paid API, vector database, or browser is required. The Actor analyzes telemetry supplied in the input, keeping compute and platform cost low.

Input

rag_logs is required. Each row can be a JSON object or a JSON string. Every valid row needs at least:

• stage
• latency_ms

For stronger analysis, add request_id, status, or success.

Example

{
  "rag_logs": [
    {"request_id": "req-1", "stage": "embedding", "latency_ms": 82, "status": "ok"},
    {"request_id": "req-1", "stage": "retrieval", "latency_ms": 238, "status": "ok"},
    {"request_id": "req-1", "stage": "reranking", "latency_ms": 144, "status": "ok"},
    {"request_id": "req-1", "stage": "generation", "latency_ms": 1180, "status": "ok"}
  ],
  "expected_stages": ["embedding", "retrieval", "reranking", "generation"],
  "end_to_end_sla_ms": 2200,
  "stage_sla_ms": {
    "retrieval": 350,
    "generation": 1500
  }
}

Arbitrary stages

The profiler is not limited to the classic retrieval/embedding/generation trio. You can instrument stages such as:

• query rewriting
• cache lookup
• hybrid search
• graph traversal
• metadata filtering
• reranking
• prompt assembly
• guardrails
• model generation
• citation validation

Stage names are preserved exactly in the output.

End-to-end measurement

When spans include request_id, the Actor sums stage latency per request and reports measured request-level p50/p90/p95/p99 and SLO breach rate.

When request IDs are absent, the Actor reports only a clearly labelled additive estimate based on stage averages. It does not pretend that an inferred average is a measured end-to-end percentile.

Output

A run emits one default-dataset item containing the complete audit:

• stage_profiles
• bottleneck_ranking
• end_to_end
• recommendations
• instrumentation_gaps
• evidence confidence
• method note explaining measured versus estimated metrics

The same result is written to the OUTPUT key-value-store record.

Limitations

Responsible interpretation

Tail percentiles are unstable with very small samples. The Actor flags low-sample stages and reports confidence rather than dressing four traces in a lab coat and calling them production science.

Optimization suggestions are hypotheses to benchmark, not claims that a specific infrastructure change will always improve performance.

Data provenance and privacy

All calculations use only the trace rows you provide. The Actor does not send your telemetry to external model providers. Avoid including prompts, retrieved documents, secrets, or personal data when stage metadata and timings are sufficient.

Pricing

Pricing is pay per completed audit: one successfully emitted performance audit equals one billable default-dataset item. The Store price is activated only after the factory checks real Apify validation-run cost and margin.

Search terms

RAG latency profiler, RAG observability, RAG bottleneck analyzer, RAG performance benchmark, retrieval latency, LLM latency audit, RAG p95 p99, vector search performance, AI pipeline SLO, RAG monitoring.

Commercial intelligence layer

Gen2 adds baseline latency regression, projected request volume, optional infrastructure-cost impact, confidence scoring and a GO / WARN / BLOCK release decision on top of the existing stage percentiles, bottleneck ranking and SLO evidence.