LLM Latency & Cost Monitor

Probes real LLM provider APIs to measure live wall-clock latency and reachability, and pairs each result with a versioned reference pricing config so you can compare providers for routing / cost optimization.

This actor performs genuine network I/O. Every latency value is a real performance.now() measurement of an actual HTTPS round-trip to the provider's API host. Nothing is randomized, simulated, or hardcoded. Pricing is the one explicitly non-live field, and it is clearly labelled as a reference config in every output record (see Pricing).

What it does

For each requested provider, the actor runs one of two real probes:

1. Authenticated probe — if an API key is available for that provider (from the input or environment), it sends a minimal real chat/completions request (max_tokens: 1, prompt "ping") and measures end-to-end latency plus the real HTTP status code.
2. Unauthenticated probe — if no key is present, it still does a real probe: a plain HTTPS GET to the provider's API host (e.g. /v1/models). This returns a real round-trip time and typically an HTTP 401/403 (expected without a key), which confirms reachability and measures network latency without spending any API credits.

Probes for all providers run concurrently. Each result is pushed to the dataset, and a roll-up is written to the SUMMARY key in the default key-value store.

Supported providers

OpenAI, Anthropic, Google, DeepSeek, Mistral, Groq.

An unrecognized provider name is not fabricated — it is returned with status: "unsupported-provider", latencyMs: null, and an explanatory error.

Input

All inputs are optional.

Field	Type	Default	Description
providers	array of strings	["OpenAI","Anthropic","Google","DeepSeek"]	Providers to probe. Supported names listed above. Blank/whitespace entries are ignored.
apiKeys	object (secret)	none	Map of provider name to API key, e.g. {"OpenAI":"sk-..."}. When a key is present for a provider, that provider gets an authenticated probe (a real minimal chat request). Marked secret.
models	object	none	Map of provider name to model id to probe, e.g. {"OpenAI":"gpt-4o-mini"}. Only used for authenticated probes; otherwise a cheap default model per provider is used.
timeoutMs	integer	15000	Hard per-probe timeout in milliseconds. A probe that exceeds it is reported with status: "timeout" and its real elapsed time. Non-positive / non-numeric values fall back to the default.

API keys via environment variables

Keys can also be supplied as environment variables instead of (or in addition to) apiKeys: OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY (or GEMINI_API_KEY), DEEPSEEK_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY.

The output records where each key came from in the keySource field (input.apiKeys, input.<ENV>, or env.<ENV>).

Example input

{
  "providers": ["OpenAI", "Anthropic", "Groq"],
  "timeoutMs": 12000,
  "apiKeys": {
    "OpenAI": "sk-..."
  }
}

With this input, OpenAI is probed authenticated (real chat request), while Anthropic and Groq are probed unauthenticated (real host latency).

Output

One dataset record per provider. Example (unauthenticated probe, no key supplied):

{
  "provider": "OpenAI",
  "latencyMs": 224,
  "inputTokenCostPerMillion": 0.15,
  "outputTokenCostPerMillion": 0.60,
  "pricingSource": "reference-config",
  "pricingConfigVersion": "2026-06-21",
  "pricingModel": "gpt-4o-mini",
  "timestamp": "2026-06-24T11:44:56.647Z",
  "status": "reachable-unauthorized",
  "probeMode": "unauthenticated",
  "probedModel": null,
  "httpStatus": 401,
  "keySource": null,
  "error": null
}

Field reference

Field	Meaning
provider	Provider name that was probed.
latencyMs	Real measured wall-clock latency of the probe, in ms (performance.now). null only when no probe could run (e.g. unsupported provider).
inputTokenCostPerMillion / outputTokenCostPerMillion	USD per 1,000,000 tokens, from the reference pricing config. null if no price entry exists for the provider.
pricingSource	Always "reference-config".
pricingConfigVersion	Date/version of the pricing reference used.
pricingModel	The specific model the reference price applies to.
timestamp	ISO 8601 time the measurement was taken.
status	Derived from the real result: online (2xx), reachable-unauthorized (401/403), rate-limited (429), server-error (5xx), http-<code>, timeout, error, or unsupported-provider.
probeMode	authenticated, unauthenticated, or none (unsupported provider).
probedModel	Model id used for an authenticated probe; null for unauthenticated probes.
httpStatus	Raw HTTP status code, or null on connection error/timeout.
keySource	Where the key came from, or null if unauthenticated.
error	Error message on failure, else null.

A SUMMARY object is also written to the default key-value store with probedAt, providerCount, pricingConfigVersion, anyAuthenticated, and the full records array.

Pricing is a reference config, not a live feed

Token prices come from a maintained, versioned config (src/pricing.js), captured from each provider's public pricing page on the lastUpdated date. They are not fetched live. Every record carries pricingSource: "reference-config" and pricingConfigVersion so you always know the prices are reference values, not a real-time quote. Update src/pricing.js (and its version/lastUpdated) when provider prices change. Latency, by contrast, is always measured live on every run.

Limitations

• Pricing is not live — see above. Verify against the provider's current pricing page before making cost decisions.
• Latency depends on where the actor runs — measured round-trip time reflects the network path between the Apify run's region and the provider, not the latency you would see from your own infrastructure.
• Unauthenticated probes measure host reachability + network latency, not model inference time. Only an authenticated probe (with a key) reflects real end-to-end chat-completion latency, and even then it is a single 1-token request, not a benchmark.
• Single sample per run — each provider is probed once. For trend data, schedule the actor and aggregate runs.
• A probe failure (timeout, DNS, connection reset) is reported honestly in status/error; it does not abort the other providers.

Local development

npm install
# write an input first:
#   storage/key_value_stores/default/INPUT.json
npx apify run --purge

Requires Node.js 18+ (uses the global fetch and AbortController).