# Agent Trace Efficiency Auditor (`firstrate/agent-trace-efficiency-auditor`) Actor

Audit AI-agent traces for token-heavy context, retries, model switching, repeated tool transformations, and human-interruption pressure. Returns the smallest matched-replay experiments needed to verify whether cognition can be removed safely — without inventing savings claims.

- **URL**: https://apify.com/firstrate/agent-trace-efficiency-auditor.md
- **Developed by:** [First Rate](https://apify.com/firstrate) (community)
- **Categories:** AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$10.00 / 1,000 trace auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Agent Trace Efficiency Auditor

Audit AI-agent traces for **wasted-cognition candidates** without pretending that lower token counts or fewer calls automatically mean a better agent.

This Actor accepts generic trace/run JSON, flat OpenTelemetry-style spans, and AgentTrace-style traces with `spans[]` + `llm_steps[]`. It returns one audit row per trace. It is deterministic, uses no LLM, calls no external API, and is designed for CI, agent observability, routing experiments, and optimization-control workflows.

### What it detects

- high input-token model calls worth testing with smaller context projections;
- model switching that may or may not be earning its complexity;
- error/failure spans that may create retry overhead;
- repeated tool-name structure;
- **exact repeated tool + normalized-input transformations** that may be candidates for caching or deterministic compilation;
- human approval/review/handoff spans that may deserve sovereignty-vs-routine classification.

### What it deliberately does NOT claim

A structural trace audit cannot prove that a cheaper configuration is better.

- same tool sequence **does not** imply same transformation;
- fewer model switches **does not** imply better routing;
- fewer tokens **does not** imply preserved task quality;
- fewer human approvals **does not** imply preserved human sovereignty;
- projected savings are not settled savings.

Every output therefore contains optimization **hypotheses and replay tests**, with optimization authority disabled until a matched workload preserves terminal outcomes and protected boundaries.

### Public-trace evidence

The current parser was replayed against **200 public NL2Bash agent traces** from the Apache-2.0 `pagarsky/agent-trace` dataset. That sample contained 516 model steps, 465 tool calls, and about 1.40M observed tokens.

The auditor produced:

- `failure_retry_overhead` on 110/200 traces with observed failures;
- `crystallization_candidate` on 28/200 traces with repeated exact tool + normalized-input signatures;
- `structural_recurrence_only` on 23/200 traces where tool recurrence existed without exact transformation recurrence;
- `context_projection_headroom` on only 1/200 traces at a 10,000-input-token threshold;
- `no_obvious_headroom` on 82/200 traces.

There were **0 optimization-authority regressions and 0 unverified-savings promotions**. These measurements show that the auditor can distinguish several trace structures and abstain when its heuristics do not expose a strong signal. They do **not** prove customer demand, realized savings, or generalization to every agent runtime.

### Input

You can provide grouped traces:

```json
{
  "traces": [
    {
      "traceId": "checkout-agent-42",
      "spans": [
        {
          "name": "chat",
          "attributes": {
            "gen_ai.request.model": "gpt-example",
            "gen_ai.usage.input_tokens": 14000,
            "gen_ai.usage.output_tokens": 500
          }
        },
        {
          "name": "tool:inventory",
          "attributes": {
            "gen_ai.tool.name": "inventory",
            "gen_ai.tool.call.arguments": { "sku": "ABC-123" }
          }
        }
      ]
    }
  ]
}
```

Or flat spans carrying `traceId`, `trace_id`, `sessionId`, or `session_id`.

The parser recognizes common attribute families including `gen_ai.*`, `llm.*`, several OpenInference-style fields, and AgentTrace-native top-level `tool_name`, `tool_input`, `exit_code`, plus `llm_steps` token telemetry. Unknown fields are ignored rather than guessed.

### Output

Each trace audit includes:

- observed model/tool call counts;
- observed input/output tokens where present;
- model switches;
- error spans;
- human-boundary spans;
- high-input-token calls;
- repeated tool shapes;
- exact repeated transformation signatures;
- evidence-backed optimization experiments;
- an explicit authority boundary explaining why replay is still required.

### Typical workflow

1. Export traces from your agent runtime or observability platform.
2. Run this Actor.
3. Read the highest-evidence optimization hypothesis.
4. Replay the **same task** with the proposed cheaper/simpler configuration.
5. Preserve terminal success, safety/authority boundaries, and required quality.
6. Only then call the change an optimization.

### Pricing

The public Actor uses Apify **pay per event**. One successfully emitted trace-group audit is the `trace-audited` event, currently configured at **$0.01**.

The implementation is intentionally low-compute: 128 MB default memory, no browser, no proxies, no LLM, and no third-party services.

### Privacy

The Actor only processes the trace data supplied to its run and writes its own result dataset/output. Avoid supplying secrets or sensitive message content when structural telemetry is sufficient. Tool inputs are reduced to short hashes for recurrence detection rather than reproduced in output.

### Method

The core method is derived from a broader optimization-control discipline: preserve the real objective, activate only decision-capable evidence, identify the cheapest useful experiment, verify changes causally, and crystallize repetition only after equivalence is established.

The customer does not need to adopt that methodology. The Actor simply returns a bounded, machine-readable audit.

# Actor input Schema

## `traces` (type: `array`):

One or more trace/run objects. Each item may contain a spans array, or may itself be an OpenTelemetry-style span.

## `spans` (type: `array`):

Alternative flat span input. Supports common gen\_ai.*, llm.*, and OpenInference-style attributes.

## `highInputTokenThreshold` (type: `integer`):

Only used to label token-heavy calls for review; it does not prove waste.

## `maxItems` (type: `integer`):

Safety cap on the number of trace groups analyzed in one run.

## Actor input object example

```json
{
  "highInputTokenThreshold": 10000,
  "maxItems": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

One machine-readable audit item per trace group, including observed structure, bounded optimization hypotheses, and authority boundaries.

## `summary` (type: `string`):

Aggregate trace/model/tool/token/error/human-boundary counts for the run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("firstrate/agent-trace-efficiency-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("firstrate/agent-trace-efficiency-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call firstrate/agent-trace-efficiency-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,firstrate/agent-trace-efficiency-auditor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BKGUeMqIiFcaELH8d/builds/ELpPoMau7jzC6ZtXI/openapi.json