# Multi-Model LLM Compare - GPT, Claude, Gemini & Llama (`seemuapps/multi-model-llm-compare`) Actor
Run one prompt across many language models at once and get every response side by side, with token counts, cost, and latency per model in a single dataset.
- **URL**: https://apify.com/seemuapps/multi-model-llm-compare.md
- **Developed by:** [Andrew](https://apify.com/seemuapps) (community)
- **Categories:** AI, Agents, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $1.00 / 1,000 usage units
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Multi-Model LLM Compare
Send one prompt to many language models at once and get every answer side by side in a single table. Compare GPT, Claude, Gemini, Llama, DeepSeek, Kimi, Qwen, GLM, and 300+ more models on quality, speed, and cost, without juggling separate accounts or API keys.
### What you get
One row per model, with:
- The full text response from each model
- Prompt tokens, completion tokens, and total tokens
- Latency in milliseconds and the provider that served it
- Finish reason, status, and any error message
### Use cases
- Pick the best and cheapest model for a task before you commit to it in production
- Benchmark answer quality, speed, and price across providers on your own prompts
- A/B test a system prompt across models
- Build evaluation datasets for prompt engineering
### How to use
1. Enter your **Prompt**
2. List the **Models** to compare, for example `openai/gpt-4o-mini` and `anthropic/claude-haiku-4.5`. Leave it empty to use a default cross-provider set
3. Optionally add a **System prompt**, a **Temperature**, and a **Max output tokens** cap
4. Run the actor. Each model becomes one row in the **Dataset** tab, ready to sort by cost, speed, or token count
### Input example
```json
{
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"models": [
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"google/gemini-2.5-flash",
"meta-llama/llama-3.3-70b-instruct"
],
"maxTokens": 300
}
````
### Output example
```json
{
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"model": "google/gemini-2.5-flash",
"provider": "Google",
"response": "Imagine you have two special coins that always land on the same side...",
"promptTokens": 15,
"completionTokens": 65,
"totalTokens": 80,
"latencyMs": 1001,
"finishReason": "stop",
"status": "success",
"error": null
}
```
### Pricing
This actor is **usage-based** — you pay in proportion to how much work each model call actually does, not a flat fee per response.
- **Short answers from small models cost very little; long answers from premium models cost more.** A quick comparison across a few small models is typically a fraction of a cent each.
- **You are only charged for models that successfully respond.** A model that errors out costs you nothing.
- **The token counts in each row** (prompt, completion, total) show how much each model produced, so you can see what drives your spend and pick the most efficient model for your task.
- **Tip:** set **Max output tokens** to cap how much each model can generate and keep your spend predictable — billing scales with the amount each model produces.
#### What it costs in practice
Price depends on which models you pick and how long their answers are. As a rough guide, here's the approximate cost **per model response**:
| Answer length | Small & fast models
(Gemini Flash, GPT-4o mini, Llama, DeepSeek) | Mid models
(Claude Haiku, Kimi, GLM) | Premium models
(Claude Sonnet/Opus, GPT-5, Gemini Pro) |
|---|---|---|---|
| Short (~150 words) | under $0.005 | ~$0.005–0.01 | ~$0.02–0.05 |
| Long (~600 words) | ~$0.01–0.02 | ~$0.02–0.04 | ~$0.06–0.15 |
A few example runs:
- **Quick check** — the 4 default models (Gemini Flash, GPT-4o mini, Claude Haiku, Llama) on a short answer → **about $0.01–0.02 for the whole run**
- **One premium model, detailed answer** — Claude Opus on a ~600-word response → **about $0.12–0.15**
- **Broad comparison** — 6 mixed models on medium answers → **roughly $0.10–0.20 total**
These are approximate — your actual cost tracks the token counts shown in each row. Lower **Max output tokens** to bring costs down, and remember you're only billed for models that actually respond.
### Free tier
You can try this actor for free, with limits:
- Up to **3 models** per run
- Up to **512 output tokens** per model
- **3 runs per day**, with a **30-minute cooldown** between runs
Upgrade to a paid plan to compare up to 12 models at once, set output length up to 8192 tokens, and run without daily limits or cooldowns.
### Notes
- Each model runs independently, so one failing model never blocks the others. Failed models return a row with `status` set to `error` and a short message.
# Actor input Schema
## `prompt` (type: `string`):
The question or instruction to send to every model.
## `models` (type: `array`):
Pick the models to compare. Leave empty to use a default cross-provider set (Gemini, GPT, Claude, Llama).
## `systemPrompt` (type: `string`):
Optional system instruction applied to every model, for example 'You are a concise expert assistant.'
## `temperature` (type: `number`):
Optional sampling temperature from 0 to 2. Leave empty to use each model's default.
## `maxTokens` (type: `integer`):
Maximum tokens each model may generate. Lower values cap cost per response. Defaults to 1024.
## Actor input object example
```json
{
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"models": [
"google/gemini-2.5-flash",
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"meta-llama/llama-3.3-70b-instruct"
],
"maxTokens": 1024
}
```
# Actor output Schema
## `results` (type: `string`):
One record per model. Fields: prompt, model, provider, response, promptTokens, completionTokens, totalTokens, latencyMs, finishReason, status, error.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"models": [
"google/gemini-2.5-flash",
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"meta-llama/llama-3.3-70b-instruct"
],
"systemPrompt": ""
};
// Run the Actor and wait for it to finish
const run = await client.actor("seemuapps/multi-model-llm-compare").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"models": [
"google/gemini-2.5-flash",
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"meta-llama/llama-3.3-70b-instruct",
],
"systemPrompt": "",
}
# Run the Actor and wait for it to finish
run = client.actor("seemuapps/multi-model-llm-compare").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"prompt": "Explain quantum entanglement to a 10 year old in 3 sentences.",
"models": [
"google/gemini-2.5-flash",
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"meta-llama/llama-3.3-70b-instruct"
],
"systemPrompt": ""
}' |
apify call seemuapps/multi-model-llm-compare --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=seemuapps/multi-model-llm-compare",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Multi-Model LLM Compare - GPT, Claude, Gemini & Llama",
"description": "Run one prompt across many language models at once and get every response side by side, with token counts, cost, and latency per model in a single dataset.",
"version": "1.0",
"x-build-id": "jqUULLVUoxFOGjXk4"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/seemuapps~multi-model-llm-compare/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-seemuapps-multi-model-llm-compare",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/seemuapps~multi-model-llm-compare/runs": {
"post": {
"operationId": "runs-sync-seemuapps-multi-model-llm-compare",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/seemuapps~multi-model-llm-compare/run-sync": {
"post": {
"operationId": "run-sync-seemuapps-multi-model-llm-compare",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"prompt"
],
"properties": {
"prompt": {
"title": "Prompt",
"type": "string",
"description": "The question or instruction to send to every model."
},
"models": {
"title": "Models",
"type": "array",
"description": "Pick the models to compare. Leave empty to use a default cross-provider set (Gemini, GPT, Claude, Llama).",
"items": {
"type": "string",
"enum": [
"anthropic/claude-opus-4.8",
"anthropic/claude-sonnet-4.6",
"anthropic/claude-haiku-4.5",
"openai/gpt-5",
"openai/gpt-5-mini",
"openai/gpt-4.1",
"openai/gpt-4o-mini",
"google/gemini-2.5-pro",
"google/gemini-2.5-flash",
"meta-llama/llama-3.3-70b-instruct",
"deepseek/deepseek-r1",
"deepseek/deepseek-chat",
"moonshotai/kimi-k2",
"qwen/qwen3-max",
"z-ai/glm-4.6"
],
"enumTitles": [
"Claude Opus 4.8 (most capable)",
"Claude Sonnet 4.6 (flagship)",
"Claude Haiku 4.5 (fast)",
"GPT-5 (flagship)",
"GPT-5 mini (fast, low cost)",
"GPT-4.1",
"GPT-4o mini (fast, low cost)",
"Gemini 2.5 Pro (flagship)",
"Gemini 2.5 Flash (fast, low cost)",
"Llama 3.3 70B (open source)",
"DeepSeek R1 (reasoning)",
"DeepSeek Chat (low cost)",
"Kimi K2 (Moonshot)",
"Qwen3 Max (Alibaba)",
"GLM-4.6 (Z.ai)"
]
},
"default": [
"google/gemini-2.5-flash",
"openai/gpt-4o-mini",
"anthropic/claude-haiku-4.5",
"meta-llama/llama-3.3-70b-instruct"
]
},
"systemPrompt": {
"title": "System prompt (optional)",
"type": "string",
"description": "Optional system instruction applied to every model, for example 'You are a concise expert assistant.'"
},
"temperature": {
"title": "Temperature (optional)",
"minimum": 0,
"maximum": 2,
"type": "number",
"description": "Optional sampling temperature from 0 to 2. Leave empty to use each model's default."
},
"maxTokens": {
"title": "Max output tokens",
"minimum": 1,
"maximum": 8192,
"type": "integer",
"description": "Maximum tokens each model may generate. Lower values cap cost per response. Defaults to 1024.",
"default": 1024
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```