# LLM VRAM Calculator: GPU Fit for Hugging Face Models (`punkjazz-labs/llm-vram-calculator`) Actor

Estimate GPU memory for any Hugging Face LLM: weights, KV cache and total VRAM per quantisation (FP16, FP8, INT8, GGUF, AWQ, GPTQ), MoE active params, and fit on RTX, A100, H100, H200, MI300X, Mac and DGX Spark. Pay per result, MCP ready.

- **URL**: https://apify.com/punkjazz-labs/llm-vram-calculator.md
- **Developed by:** [Punkjazz Labs](https://apify.com/punkjazz-labs) (community)
- **Categories:** AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## LLM VRAM Calculator: GPU Fit for Hugging Face Models

Find out **how much GPU memory (VRAM) a Hugging Face LLM needs** and **which GPUs or Macs it fits on**, before you download 50 GB of weights or rent a cloud GPU. Give it model ids (or a search query) and get, for every quantisation you pick: weights, KV cache at your context length, runtime overhead, total VRAM, and fit, headroom and minimum GPU count across 19 common GPUs and devices.

- **Reads the real model.** Parameter counts come from the model's safetensors (or GGUF) metadata and the KV-cache maths from its `config.json` (layers, KV heads, head size, sliding-window layers, MLA). No guessing from the model name.
- **Mixture-of-experts aware.** Reports total and active parameters for Mixtral, Qwen3 MoE, DeepSeek, gpt-oss and similar.
- **All the usual quantisations.** FP32, FP16, BF16, FP8, INT8, GGUF Q8_0 / Q6_K / Q5_K_M / Q4_K_M / Q3_K_M, INT4, AWQ, GPTQ, MXFP4.
- **Fit on real hardware.** RTX 3090, 4090, 5090, L40S, A100 40/80 GB, H100 80 GB, H200, AMD MI300X, NVIDIA DGX Spark, and Apple M-series Macs from 16 GB to 512 GB, plus any custom GPU size.
- **Gated models supported.** Add your own Hugging Face read token for Llama, Gemma and other gated repos. It is stored as a secret input and never logged.
- **Fast and cheap.** No browser, only the Hugging Face API. A run takes a few seconds and costs $0.003 per estimate row.

All numbers are **estimates** for inference (not training). See [How the estimate works](#how-the-estimate-works) for the exact formulas.

### What you get

One row per model × quantisation:

| Field | Example | Notes |
|---|---|---|
| `modelId` | Qwen/Qwen3-30B-A3B | Hugging Face model id |
| `quantization` | q4_k_m | With `bitsPerWeight` (4.89) and a readable label |
| `totalParamsB` / `activeParamsB` | 30.53 / 3.35 | Billions. Active < total for MoE models |
| `weightsGB` | 18.66 | Weights at this precision |
| `kvCacheGB` | 3.22 | KV cache at `contextLength` × `batchSize` |
| `overheadGB` | 1.84 | Runtime context, activations, fragmentation |
| `totalVramGB` | 23.73 | What one device needs |
| `smallestSingleDevice` | NVIDIA RTX 4090 24 GB | Smallest checked device it fits on alone |
| `devices[]` | `{ "device": "rtx-4090", "fits": true, "headroomGB": 2.04, "minCount": 1 }` | Per device: usable GB, fit, headroom, GPUs needed with tensor parallelism |
| `attention`, `layers`, `kvHeads`, `headDim`, `attentionLayout` | GQA, 48, 4, 128, "48 full_attention" | What the KV cache maths used |
| `paramSource` | safetensors | `safetensors`, `gguf` or `config-estimate` |

Sizes are decimal GB (1 GB = 10^9 bytes).

### Quick start

1. Click **Try for free** (or **Start**) on this page.
2. Leave the input as it is, or paste the models you care about into **Hugging Face model ids**.
3. Pick quantisations and a context length, press **Start**, and read the table, or download JSON, CSV or Excel.

An empty input `{}` sizes `Qwen/Qwen3-8B` at FP16, INT8 and Q4_K_M with an 8,192-token context.

#### Example input

```json
{
  "modelIds": ["Qwen/Qwen3-30B-A3B", "openai/gpt-oss-20b", "meta-llama/Llama-3.1-8B-Instruct"],
  "quantizations": ["bf16", "fp8", "q4_k_m"],
  "contextLength": 32768,
  "targetDevices": ["rtx-4090", "rtx-5090", "h100-80gb", "apple-64gb"]
}
```

#### Example output (one row from a real run, 2 Nov 2026)

```json
{
  "modelId": "Qwen/Qwen3-30B-A3B",
  "modelUrl": "https://huggingface.co/Qwen/Qwen3-30B-A3B",
  "architecture": "qwen3_moe",
  "quantization": "q4_k_m",
  "quantizationLabel": "GGUF Q4_K_M",
  "bitsPerWeight": 4.89,
  "totalParamsB": 30.53,
  "activeParamsB": 3.35,
  "isMoE": true,
  "experts": 128,
  "expertsPerToken": 8,
  "attention": "GQA",
  "layers": 48,
  "kvHeads": 4,
  "headDim": 128,
  "attentionLayout": "48 full_attention",
  "contextLength": 32768,
  "batchSize": 1,
  "kvCacheType": "fp16",
  "kvCacheKBPerToken": 98.3,
  "weightsGB": 18.66,
  "kvCacheGB": 3.22,
  "overheadGB": 1.84,
  "totalVramGB": 23.73,
  "smallestSingleDevice": "NVIDIA RTX 4090 24 GB",
  "fitsOnSingleDevice": [
    "NVIDIA RTX 4090 24 GB",
    "NVIDIA RTX 5090 32 GB",
    "Apple M-series 64 GB unified",
    "NVIDIA H100 80 GB"
  ],
  "devices": [
    {
      "device": "rtx-4090",
      "name": "NVIDIA RTX 4090 24 GB",
      "usableGB": 25.77,
      "fits": true,
      "headroomGB": 2.04,
      "minCount": 1
    },
    {
      "device": "rtx-5090",
      "name": "NVIDIA RTX 5090 32 GB",
      "usableGB": 34.36,
      "fits": true,
      "headroomGB": 10.63,
      "minCount": 1
    },
    {
      "device": "h100-80gb",
      "name": "NVIDIA H100 80 GB",
      "usableGB": 85.9,
      "fits": true,
      "headroomGB": 62.17,
      "minCount": 1
    },
    {
      "device": "apple-64gb",
      "name": "Apple M-series 64 GB unified",
      "usableGB": 51.54,
      "fits": true,
      "headroomGB": 27.81,
      "minCount": 1
    }
  ],
  "paramSource": "safetensors",
  "maxContextLength": 40960,
  "notes": [],
  "isEstimate": true
}
```

The same run also returned BF16 and FP8 rows for Qwen3-30B-A3B and three rows for gpt-oss-20b. `meta-llama/Llama-3.1-8B-Instruct` was listed under `errors` in `OUTPUT` because it is gated and no token was given, and it was not charged.

The `OUTPUT` record in the run's key-value store lists models that could not be sized, warnings and the assumptions used.

### Input

All fields are optional.

| Field | Default | What it does |
|---|---|---|
| `modelIds` | `["Qwen/Qwen3-8B"]` if nothing else is given | Model ids like `org/name`, or huggingface.co links |
| `searchQuery` | none | Also size the most downloaded text-generation models matching this text |
| `searchLimit` | 5 | How many search results to size (1 to 50) |
| `quantizations` | `["fp16", "int8", "q4_k_m"]` | Any of fp32, fp16, bf16, fp8, int8, q8_0, q6_k, q5_k_m, q4_k_m, q3_k_m, int4, awq, gptq, mxfp4 (q4, q8, 8bit also work) |
| `contextLength` | 8192 | Tokens in the KV cache per sequence. `0` = the model's maximum |
| `batchSize` | 1 | Concurrent sequences, each with its own KV cache |
| `kvCacheType` | fp16 | fp16, bf16, fp8, q8_0 or q4_0 |
| `targetDevices` | all built-ins | Built-in ids, part of a name (`4090`), or a custom `"Name=GB"` such as `"RTX A6000=48"` |
| `hfToken` | none | Hugging Face read token for gated or private models (secret) |

Built-in device ids: `rtx-3090`, `rtx-4090`, `rtx-5090`, `l40s`, `a100-40gb`, `a100-80gb`, `h100-80gb`, `h200`, `mi300x`, `dgx-spark`, `apple-16gb`, `apple-24gb`, `apple-32gb`, `apple-48gb`, `apple-64gb`, `apple-96gb`, `apple-128gb`, `apple-192gb`, `apple-512gb`.

### How the estimate works

These are the formulas, so you can check or adjust them.

- **Parameters**: `safetensors.total` from the Hugging Face API. For GGUF-only repos, the GGUF parameter count. If neither is published, an estimate from `config.json` (embeddings, attention, MLP and experts).
- **Weights** = parameters × bits per weight ÷ 8. Bits per weight: FP32 32, FP16/BF16 16, FP8 and INT8 8, Q8_0 8.5, Q6_K 6.56, Q5_K_M 5.69, Q4_K_M 4.89, Q3_K_M 3.91, INT4/AWQ/GPTQ/MXFP4 4.25 (4-bit plus group-128 scales). Every weight is assumed to use the chosen precision; real files keep some tensors (embeddings, norms) at higher precision, so they can be slightly larger.
- **KV cache** = 2 (K and V) × KV heads × head size × bytes per element × tokens cached, summed over layers, × batch size.
  - Full-attention layers cache the whole context. Sliding-window layers (gpt-oss, Gemma 2/3, some Mistral) cache at most the window. Linear-attention and state-space layers (Qwen3-Next, hybrids) are counted as zero.
  - Multi-head latent attention (DeepSeek V2/V3, Kimi K2) caches `kv_lora_rank + qk_rope_head_dim` values per layer instead.
  - Example: Llama-3.1-8B has 32 layers, 8 KV heads and head size 128, so FP16 needs 2 × 32 × 8 × 128 × 2 bytes = 128 KiB per token, or 1.07 GB at 8,192 tokens.
- **Overhead** = 0.75 GB (CUDA/Metal context, allocator, small buffers) + 5% of weights and KV cache (activations, fragmentation).
- **Total** = weights + KV cache + overhead.
- **Active parameters (MoE)** = total − (experts − experts per token) × expert size × MoE layers, with expert size = 3 × hidden size × expert intermediate size. This is the compute per token; **all experts still have to be in memory**, so the memory figures use total parameters.
- **Usable memory**: GPUs are sold in binary units, so a "24 GB" RTX 4090 has 24 GiB = 25.77 GB. Discrete GPUs: 100% usable. Apple Macs: macOS lets the GPU use about 67% of unified memory up to 36 GB and 75% above (you can raise it with `sysctl iogpu.wired_limit_mb`). DGX Spark: 90% of 128 GB.
- **Fit** = total ≤ usable memory. `headroomGB` is usable memory minus total (negative means short by that much).
- **Minimum GPUs (`minCount`)**: the smallest of 1, 2, 4 or 8 identical GPUs where each holds (weights + KV cache) ÷ n plus overhead, as with tensor parallelism in vLLM or SGLang. `null` means more than 8 GPUs (multi-node) or a device that is not normally combined (Macs, DGX Spark).

Not included: training or fine-tuning memory (optimizer states, gradients), CPU or disk offload, speculative-decoding draft models, vision encoders' activations, and engine-specific pre-allocation (vLLM reserves `gpu_memory_utilization` of the card, 90% by default, and fills the rest with KV cache).

### Pricing

This Actor uses **pay-per-event** pricing:

| Event | Price |
|---|---|
| Estimate returned (one model × quantisation row) | $0.003 |
| Run start (per GB of memory, default run uses 256 MB = one event) | $0.005 |

Example: 5 models × 3 quantisations = 15 rows = $0.045 + $0.005 = **$0.05**. Models that fail (gated without a token, not found) are not charged. Set **Max total charge** on the run and the Actor stops before it goes over.

### Use with AI agents and the Apify MCP server

Coding agents and chat assistants are often asked "will this model run on my GPU?" This Actor gives them a grounded answer from the model's real config instead of a guess. Pay-per-event pricing and fast runs make it a good MCP tool for Claude, ChatGPT, Cursor, n8n or LangChain through the [Apify MCP server](https://mcp.apify.com).

Add only this Actor as a tool:

```json
{
  "mcpServers": {
    "llm-vram-calculator": { "url": "https://mcp.apify.com?tools=punkjazz-labs/llm-vram-calculator" }
  }
}
```

Or run the server locally: `npx -y @apify/actors-mcp-server --tools punkjazz-labs/llm-vram-calculator`.

Example prompts:

- "Can I run Qwen3-30B-A3B on my RTX 4090 at 32k context? Which quantisation?"
- "How many H100s do I need for DeepSeek-V3 in FP8?"
- "What's the biggest Qwen3 model that fits on a 64 GB MacBook at Q4_K_M?"

### API

```bash
curl -X POST "https://api.apify.com/v2/acts/punkjazz-labs~llm-vram-calculator/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" -H 'Content-Type: application/json' \
  -d '{"modelIds": ["Qwen/Qwen3-30B-A3B"], "quantizations": ["bf16", "q4_k_m"], "contextLength": 32768}'
```

### Use cases

- Choosing a GPU, cloud instance or Mac for a model before you buy or rent
- Picking the quantisation and context length that fit the hardware you already have
- Comparing many models at once for a self-hosting shortlist
- Capacity planning for vLLM, SGLang, TGI, llama.cpp, Ollama or LM Studio deployments
- Giving an AI agent a reliable "will it fit?" tool

### FAQ

**How accurate is it?** Weights are usually within a few percent of the real file size. Total memory depends on your inference engine and its settings, so treat the result as a planning estimate and keep some headroom.

**Why does my Llama or Gemma model fail?** Those repos are gated. Accept the licence on the model page on Hugging Face, create a read token at huggingface.co/settings/tokens and paste it into **Hugging Face token**. The token is a secret input: it is encrypted, only sent to huggingface.co and never logged or written to the results.

**Does it download the model?** No. It reads only the API metadata and `config.json`, a few kilobytes per model.

**Why is a MoE model's memory based on total parameters?** All experts must be loaded, even though each token only uses a few of them. `activeParamsB` tells you the compute per token, which drives speed, not memory.

**Does it work for GGUF repos?** Yes. It uses the GGUF parameter count and, if the repo has no `config.json`, the base model's config.

**Can I add my own GPU?** Yes. Add `"Name=GB"` to **Target GPUs / devices**, for example `"RTX A6000=48"`.

**Does it cover training?** No. Training and fine-tuning need memory for gradients and optimizer states, often 4 to 8 times more than inference.

**Can it rank models by which fit my GPU?** Yes. Use a search query plus `targetDevices` with your one GPU, then sort the results by `totalVramGB` or filter on `devices[0].fits`.

### Related Actors

- [UAE FTA Tax News Monitor](https://apify.com/punkjazz-labs/uae-fta-regulatory-monitor): VAT and Corporate Tax updates for UAE businesses.
- [Dubai eSupply Tenders Scraper](https://apify.com/punkjazz-labs/dubai-esupply-tenders): Dubai Government RFQs and RFPs.

### Notes

- Estimates only, not measurements. Not affiliated with Hugging Face, NVIDIA, AMD or Apple.
- Questions, missing GPUs or wrong numbers for a model: open an issue in the Issues tab on this page.

# Actor input Schema

## `modelIds` (type: `array`):

Models to size, as "org/name" or a huggingface.co link. Each model x quantisation is one billable row ($0.003). Leave empty and add a search query, or leave both empty to size Qwen/Qwen3-8B. Gated models (Llama, Gemma) need a token below.

## `searchQuery` (type: `string`):

Optional. Also size the most downloaded text-generation models on Hugging Face matching this text. Example: "llama 3.3 70b" or "gpt-oss".

## `searchLimit` (type: `integer`):

How many models to take from the search query (1 to 50), most downloaded first. Ignored without a search query.

## `quantizations` (type: `array`):

Weight precisions to compare. Supported: fp32, fp16, bf16, fp8, int8, q8_0, q6_k, q5_k_m, q4_k_m, q3_k_m, int4, awq, gptq, mxfp4 (short forms like q4, q8 and 8bit also work). One row per model per quantisation.

## `contextLength` (type: `integer`):

Tokens held in the KV cache per sequence (prompt plus output). Default 8192. Use 0 for each model's maximum context from config.json. Example: 32768.

## `batchSize` (type: `integer`):

How many sequences run at the same time, each with its own full-length KV cache. 1 for a single chat user; raise it for serving many users.

## `kvCacheType` (type: `string`):

Precision of the KV cache. FP16 is the default in vLLM, llama.cpp and Ollama; FP8 or Q8_0 roughly halves cache memory.

## `targetDevices` (type: `array`):

Optional. Limit the fit check to these devices: built-in ids (rtx-3090, rtx-4090, rtx-5090, l40s, a100-40gb, a100-80gb, h100-80gb, h200, mi300x, dgx-spark, apple-16gb to apple-512gb), part of a name ("4090"), or a custom "Name=GB". Empty checks all built-ins.

## `hfToken` (type: `string`):

Optional read token for gated or private models (Llama, Gemma and similar), after you accept the licence on Hugging Face. Stored encrypted, sent only to huggingface.co, never logged or saved in results.

## Actor input object example

```json
{
  "modelIds": [
    "Qwen/Qwen3-8B",
    "mistralai/Mixtral-8x7B-v0.1"
  ],
  "searchQuery": "qwen3 coder",
  "searchLimit": 5,
  "quantizations": [
    "fp16",
    "fp8",
    "q8_0",
    "q4_k_m"
  ],
  "contextLength": 8192,
  "batchSize": 1,
  "kvCacheType": "fp16",
  "targetDevices": [
    "rtx-4090",
    "h100-80gb",
    "apple-64gb",
    "My GPU=48"
  ]
}
```

# Actor output Schema

## `results` (type: `string`):

All VRAM estimates in the default dataset.

## `summary` (type: `string`):

JSON with row count, errors (e.g. gated models without a token), warnings, assumptions and a sample.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "modelIds": [
        "Qwen/Qwen3-8B",
        "openai/gpt-oss-20b"
    ],
    "quantizations": [
        "fp16",
        "fp8",
        "q8_0",
        "q4_k_m"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("punkjazz-labs/llm-vram-calculator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "modelIds": [
        "Qwen/Qwen3-8B",
        "openai/gpt-oss-20b",
    ],
    "quantizations": [
        "fp16",
        "fp8",
        "q8_0",
        "q4_k_m",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("punkjazz-labs/llm-vram-calculator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "modelIds": [
    "Qwen/Qwen3-8B",
    "openai/gpt-oss-20b"
  ],
  "quantizations": [
    "fp16",
    "fp8",
    "q8_0",
    "q4_k_m"
  ]
}' |
apify call punkjazz-labs/llm-vram-calculator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,punkjazz-labs/llm-vram-calculator"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1eM2LM8Po67UXzB4J/builds/lDmVgXa7LrmpLb51d/openapi.json