# LLM Provider Price and Latency Monitor (`george.the.developer/llm-provider-price-latency-monitor`) Actor Normalize model pricing, context length, capabilities, and provider routing data for LLM gateways, AI builders, and FinOps teams. - **URL**: https://apify.com/george.the.developer/llm-provider-price-latency-monitor.md - **Developed by:** [George Kioko](https://apify.com/george.the.developer) (community) - **Categories:** AI, Developer tools - **Stats:** 1 total users, 1 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## LLM Provider Price and Latency Monitor Pricing across 200+ LLMs changes weekly. Tracking it by hand stopped scaling at the 3rd FinOps spreadsheet. This actor turns public model data into a normalized JSON feed for AI builders, LLM gateways, agent platforms, model marketplaces, and finance teams that need repeatable pricing and capability snapshots. OpenRouter is the primary source because its public models API exposes canonical model IDs, pricing, context length, architecture, supported parameters, and provider routing data. Direct provider pages are used only as limited fallbacks when OpenRouter does not have a model. The actor runs in Apify Standby mode, so it can be called like a small API. It also supports batch input for scheduled snapshots. ### Quick Start Filter models that support tool calling: ```bash curl "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"mode":"models","capability":"tools","provider":"anthropic","limit":5}' ```` Benchmark a family for chat routing: ```bash curl "https://YOUR_STANDBY_URL/benchmark?model_family=claude-3&task=chat&limit=2" ``` ### Standby Endpoints | Endpoint | Purpose | Billing | | --- | --- | --- | | `GET /` | Service info and endpoint list | Free | | `GET /health` | Readiness and service info | Free | | `GET /models` | Filter model snapshots by provider, capability, price, context, and limit | `model-snapshot` per row | | `GET /model?id=` | Return one full model profile with provider routings | `model-snapshot` once | | `GET /benchmark` | Compare provider routes for a model family and task | `benchmark-row` per row | | `GET /pricing-change` | Return current v1 pricing history status | Free in v1 | | `POST /watchlist` | Enrich up to 50 requested model IDs | `model-snapshot` per returned model | All endpoints honor health check values such as `ping`, `test`, `example.com`, and URL forms like `https://example.com/status`. These short circuit to a mocked LLM shaped response and do not charge. ### Model Snapshot Schema | Field | Type | Notes | | --- | --- | --- | | `model_id` | string | Canonical OpenRouter slug when available | | `name` | string | Display name | | `provider_org` | string | Source organization such as `anthropic`, `openai`, `meta`, or `mistral` | | `providers_serving` | string array | Routes serving the model, including OpenRouter and endpoint provider tags | | `pricing_per_1m_tokens` | object | Prompt, completion, image input, and flat request prices | | `context_length` | number | Token context window | | `max_output_tokens` | number or null | Advertised max completion tokens | | `supports_vision` | boolean | Based on model architecture input modalities | | `supports_tools` | boolean | Based on `tools` or `tool_choice` support | | `supports_json_mode` | boolean | Based on `response_format` or `structured_outputs` support | | `modality` | string | `text`, `multimodal`, `image`, or `audio` | | `source` | string | Current primary source is `openrouter` | | `fetched_at` | ISO date | Snapshot time | ### Benchmark Row Schema | Field | Type | Notes | | --- | --- | --- | | `benchmark_id` | string | Stable hash of family, task, model, and providers | | `model_family` | string | Family requested, such as `claude-3` | | `task` | string | `chat`, `vision`, `tools`, or `json` | | `providers` | object array | Provider, model ID, token pricing, context, and chat pair cost | | `cheapest_provider` | string | Lowest cost route for a 500 prompt plus 500 completion pair | | `fastest_advertised_provider` | string or null | Uses OpenRouter endpoint latency when present | | `best_context_provider` | string | Route with the largest context window | | `cheapest_vs_most_expensive_multiplier` | number | Most expensive route divided by cheapest route | The `total_per_1k_chat_pair_usd` metric assumes 500 prompt tokens and 500 completion tokens. It is a simple apples to apples price for one common chat exchange. ### Pipeline ```mermaid flowchart LR A[Input or Standby request] --> B[OpenRouter models API] A --> C[Limited direct provider fetchers] B --> D[Normalize model schema] C --> D D --> E{Benchmark requested} E -->|yes| F[Provider route aggregation] E -->|no| G[Model snapshots] F --> H[Dataset and JSON response] G --> H ``` ### Pricing | Event | Price | Charged when | | --- | ---: | --- | | `actor-start` | $0.50 | Once per paid Standby request or batch run | | `model-snapshot` | $0.025 | Per enriched model profile returned | | `benchmark-row` | $0.15 | Per model family comparison row returned | Charges fire only after real work succeeds. Health checks and empty not found responses are not billed. ### Comparison | Option | Best for | Gap | | --- | --- | --- | | OpenRouter (aggregator UI, not an ingestion API) | Browsing and routing through OpenRouter | Teams still need normalized export, filtering, and Apify scheduling | | Internal FinOps spreadsheets | Manual reviews and finance notes | Slow updates, inconsistent capability fields, and weak automation | | This actor (the normalized feed) | JSON snapshots, model admission checks, and recurring cost reviews | v1 has current snapshots only, not historical deltas | ### Use Cases 1. LLM gateway routing decisions by price, context, and capability. 2. AI agent platform model admission checks before enabling a new model. 3. FinOps cost per task budgeting for weekly model spend reviews. 4. AI engineering team weekly digest of price and capability changes. 5. Third party model marketplace listing data. ### FAQ **How often should I run it?** Weekly is enough for most teams. Daily is useful when you expose model choice to paying customers. **How accurate is capability detection?** It follows OpenRouter fields. Vision uses architecture modality and input modalities. Tool and JSON support use advertised supported parameters. **Can provider data lag?** Yes. OpenRouter is the primary normalized source, and direct provider pages are only fallback checks. Treat the feed as operational data, not a contract. **Does v1 include historical pricing?** Not yet. `/pricing-change` returns a clear no history response. Schedule this actor and store snapshots if you need history today. **Can I get a refund for bad source data?** Contact the actor owner with the run ID and affected model IDs. Refunds are reviewed case by case. **How do I request coverage changes?** Send the model ID, provider URL, and desired field to the actor owner through Apify. # Actor input Schema ## `mode` (type: `string`): Batch mode to run when input is provided. ## `provider` (type: `string`): Filter by provider organization or serving route, for example anthropic, openai, together, or groq. ## `capability` (type: `string`): Filter models by capability. ## `max_price` (type: `number`): Maximum prompt or completion price in USD per 1M tokens. ## `min_context` (type: `integer`): Minimum context window in tokens. ## `id` (type: `string`): Canonical model slug for mode=model. ## `model_family` (type: `string`): Family for benchmark mode. ## `task` (type: `string`): Task profile for benchmark mode. ## `models` (type: `array`): Model IDs for mode=watchlist. Maximum 50. ## `limit` (type: `integer`): Maximum rows returned. Capped at 100. ## Actor input object example ```json { "mode": "models", "task": "chat", "limit": 25 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("george.the.developer/llm-provider-price-latency-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("george.the.developer/llm-provider-price-latency-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call george.the.developer/llm-provider-price-latency-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=george.the.developer/llm-provider-price-latency-monitor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "LLM Provider Price and Latency Monitor", "description": "Normalize model pricing, context length, capabilities, and provider routing data for LLM gateways, AI builders, and FinOps teams.", "version": "1.0", "x-build-id": "8mOscAtPl1jQjtXlF" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/george.the.developer~llm-provider-price-latency-monitor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-george.the.developer-llm-provider-price-latency-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/george.the.developer~llm-provider-price-latency-monitor/runs": { "post": { "operationId": "runs-sync-george.the.developer-llm-provider-price-latency-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/george.the.developer~llm-provider-price-latency-monitor/run-sync": { "post": { "operationId": "run-sync-george.the.developer-llm-provider-price-latency-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Mode", "enum": [ "models", "model", "benchmark", "watchlist" ], "type": "string", "description": "Batch mode to run when input is provided.", "default": "models" }, "provider": { "title": "Provider", "type": "string", "description": "Filter by provider organization or serving route, for example anthropic, openai, together, or groq." }, "capability": { "title": "Capability", "enum": [ "vision", "tools", "json", "function-calling" ], "type": "string", "description": "Filter models by capability." }, "max_price": { "title": "Max price per 1M tokens", "type": "number", "description": "Maximum prompt or completion price in USD per 1M tokens." }, "min_context": { "title": "Minimum context length", "type": "integer", "description": "Minimum context window in tokens." }, "id": { "title": "Model ID", "type": "string", "description": "Canonical model slug for mode=model." }, "model_family": { "title": "Model family", "enum": [ "gpt-4", "claude-3", "llama-3", "mixtral", "mistral" ], "type": "string", "description": "Family for benchmark mode." }, "task": { "title": "Task", "enum": [ "chat", "vision", "tools", "json" ], "type": "string", "description": "Task profile for benchmark mode.", "default": "chat" }, "models": { "title": "Watchlist models", "type": "array", "description": "Model IDs for mode=watchlist. Maximum 50.", "items": { "type": "string" } }, "limit": { "title": "Limit", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum rows returned. Capped at 100.", "default": 25 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```