LLM VRAM Calculator: GPU Fit for Hugging Face Models

Find out how much GPU memory (VRAM) a Hugging Face LLM needs and which GPUs or Macs it fits on, before you download 50 GB of weights or rent a cloud GPU. Give it model ids (or a search query) and get, for every quantisation you pick: weights, KV cache at your context length, runtime overhead, total VRAM, and fit, headroom and minimum GPU count across 19 common GPUs and devices.

• Reads the real model. Parameter counts come from the model's safetensors (or GGUF) metadata and the KV-cache maths from its config.json (layers, KV heads, head size, sliding-window layers, MLA). No guessing from the model name.
• Mixture-of-experts aware. Reports total and active parameters for Mixtral, Qwen3 MoE, DeepSeek, gpt-oss and similar.
• All the usual quantisations. FP32, FP16, BF16, FP8, INT8, GGUF Q8_0 / Q6_K / Q5_K_M / Q4_K_M / Q3_K_M, INT4, AWQ, GPTQ, MXFP4.
• Fit on real hardware. RTX 3090, 4090, 5090, L40S, A100 40/80 GB, H100 80 GB, H200, AMD MI300X, NVIDIA DGX Spark, and Apple M-series Macs from 16 GB to 512 GB, plus any custom GPU size.
• Gated models supported. Add your own Hugging Face read token for Llama, Gemma and other gated repos. It is stored as a secret input and never logged.
• Fast and cheap. No browser, only the Hugging Face API. A run takes a few seconds and costs $0.003 per estimate row.

All numbers are estimates for inference (not training). See How the estimate works for the exact formulas.

What you get

One row per model × quantisation:

Field	Example	Notes
modelId	Qwen/Qwen3-30B-A3B	Hugging Face model id
quantization	q4_k_m	With bitsPerWeight (4.89) and a readable label
totalParamsB / activeParamsB	30.53 / 3.35	Billions. Active < total for MoE models
weightsGB	18.66	Weights at this precision
kvCacheGB	3.22	KV cache at contextLength × batchSize
overheadGB	1.84	Runtime context, activations, fragmentation
totalVramGB	23.73	What one device needs
smallestSingleDevice	NVIDIA RTX 4090 24 GB	Smallest checked device it fits on alone
devices[]	{ "device": "rtx-4090", "fits": true, "headroomGB": 2.04, "minCount": 1 }	Per device: usable GB, fit, headroom, GPUs needed with tensor parallelism
attention, layers, kvHeads, headDim, attentionLayout	GQA, 48, 4, 128, "48 full_attention"	What the KV cache maths used
paramSource	safetensors	safetensors, gguf or config-estimate

Sizes are decimal GB (1 GB = 10^9 bytes).

Quick start

1. Click Try for free (or Start) on this page.
2. Leave the input as it is, or paste the models you care about into Hugging Face model ids.
3. Pick quantisations and a context length, press Start, and read the table, or download JSON, CSV or Excel.

An empty input {} sizes Qwen/Qwen3-8B at FP16, INT8 and Q4_K_M with an 8,192-token context.

Example input

{
  "modelIds": ["Qwen/Qwen3-30B-A3B", "openai/gpt-oss-20b", "meta-llama/Llama-3.1-8B-Instruct"],
  "quantizations": ["bf16", "fp8", "q4_k_m"],
  "contextLength": 32768,
  "targetDevices": ["rtx-4090", "rtx-5090", "h100-80gb", "apple-64gb"]
}

Example output (one row from a real run, 2 Nov 2026)

{
  "modelId": "Qwen/Qwen3-30B-A3B",
  "modelUrl": "https://huggingface.co/Qwen/Qwen3-30B-A3B",
  "architecture": "qwen3_moe",
  "quantization": "q4_k_m",
  "quantizationLabel": "GGUF Q4_K_M",
  "bitsPerWeight": 4.89,
  "totalParamsB": 30.53,
  "activeParamsB": 3.35,
  "isMoE": true,
  "experts": 128,
  "expertsPerToken": 8,
  "attention": "GQA",
  "layers": 48,
  "kvHeads": 4,
  "headDim": 128,
  "attentionLayout": "48 full_attention",
  "contextLength": 32768,
  "batchSize": 1,
  "kvCacheType": "fp16",
  "kvCacheKBPerToken": 98.3,
  "weightsGB": 18.66,
  "kvCacheGB": 3.22,
  "overheadGB": 1.84,
  "totalVramGB": 23.73,
  "smallestSingleDevice": "NVIDIA RTX 4090 24 GB",
  "fitsOnSingleDevice": [
    "NVIDIA RTX 4090 24 GB",
    "NVIDIA RTX 5090 32 GB",
    "Apple M-series 64 GB unified",
    "NVIDIA H100 80 GB"
  ],
  "devices": [
    {
      "device": "rtx-4090",
      "name": "NVIDIA RTX 4090 24 GB",
      "usableGB": 25.77,
      "fits": true,
      "headroomGB": 2.04,
      "minCount": 1
    },
    {
      "device": "rtx-5090",
      "name": "NVIDIA RTX 5090 32 GB",
      "usableGB": 34.36,
      "fits": true,
      "headroomGB": 10.63,
      "minCount": 1
    },
    {
      "device": "h100-80gb",
      "name": "NVIDIA H100 80 GB",
      "usableGB": 85.9,
      "fits": true,
      "headroomGB": 62.17,
      "minCount": 1
    },
    {
      "device": "apple-64gb",
      "name": "Apple M-series 64 GB unified",
      "usableGB": 51.54,
      "fits": true,
      "headroomGB": 27.81,
      "minCount": 1
    }
  ],
  "paramSource": "safetensors",
  "maxContextLength": 40960,
  "notes": [],
  "isEstimate": true
}

The same run also returned BF16 and FP8 rows for Qwen3-30B-A3B and three rows for gpt-oss-20b. meta-llama/Llama-3.1-8B-Instruct was listed under errors in OUTPUT because it is gated and no token was given, and it was not charged.

The OUTPUT record in the run's key-value store lists models that could not be sized, warnings and the assumptions used.

Input

All fields are optional.

Field	Default	What it does
modelIds	["Qwen/Qwen3-8B"] if nothing else is given	Model ids like org/name, or huggingface.co links
searchQuery	none	Also size the most downloaded text-generation models matching this text
searchLimit	5	How many search results to size (1 to 50)
quantizations	["fp16", "int8", "q4_k_m"]	Any of fp32, fp16, bf16, fp8, int8, q8_0, q6_k, q5_k_m, q4_k_m, q3_k_m, int4, awq, gptq, mxfp4 (q4, q8, 8bit also work)
contextLength	8192	Tokens in the KV cache per sequence. 0 = the model's maximum
batchSize	1	Concurrent sequences, each with its own KV cache
kvCacheType	fp16	fp16, bf16, fp8, q8_0 or q4_0
targetDevices	all built-ins	Built-in ids, part of a name (4090), or a custom "Name=GB" such as "RTX A6000=48"
hfToken	none	Hugging Face read token for gated or private models (secret)

Built-in device ids: rtx-3090, rtx-4090, rtx-5090, l40s, a100-40gb, a100-80gb, h100-80gb, h200, mi300x, dgx-spark, apple-16gb, apple-24gb, apple-32gb, apple-48gb, apple-64gb, apple-96gb, apple-128gb, apple-192gb, apple-512gb.

How the estimate works

These are the formulas, so you can check or adjust them.

• Parameters: safetensors.total from the Hugging Face API. For GGUF-only repos, the GGUF parameter count. If neither is published, an estimate from config.json (embeddings, attention, MLP and experts).
• Weights = parameters × bits per weight ÷ 8. Bits per weight: FP32 32, FP16/BF16 16, FP8 and INT8 8, Q8_0 8.5, Q6_K 6.56, Q5_K_M 5.69, Q4_K_M 4.89, Q3_K_M 3.91, INT4/AWQ/GPTQ/MXFP4 4.25 (4-bit plus group-128 scales). Every weight is assumed to use the chosen precision; real files keep some tensors (embeddings, norms) at higher precision, so they can be slightly larger.
• KV cache = 2 (K and V) × KV heads × head size × bytes per element × tokens cached, summed over layers, × batch size.
  • Full-attention layers cache the whole context. Sliding-window layers (gpt-oss, Gemma 2/3, some Mistral) cache at most the window. Linear-attention and state-space layers (Qwen3-Next, hybrids) are counted as zero.
  • Multi-head latent attention (DeepSeek V2/V3, Kimi K2) caches kv_lora_rank + qk_rope_head_dim values per layer instead.
  • Example: Llama-3.1-8B has 32 layers, 8 KV heads and head size 128, so FP16 needs 2 × 32 × 8 × 128 × 2 bytes = 128 KiB per token, or 1.07 GB at 8,192 tokens.
• Overhead = 0.75 GB (CUDA/Metal context, allocator, small buffers) + 5% of weights and KV cache (activations, fragmentation).
• Total = weights + KV cache + overhead.
• Active parameters (MoE) = total − (experts − experts per token) × expert size × MoE layers, with expert size = 3 × hidden size × expert intermediate size. This is the compute per token; all experts still have to be in memory, so the memory figures use total parameters.
• Usable memory: GPUs are sold in binary units, so a "24 GB" RTX 4090 has 24 GiB = 25.77 GB. Discrete GPUs: 100% usable. Apple Macs: macOS lets the GPU use about 67% of unified memory up to 36 GB and 75% above (you can raise it with sysctl iogpu.wired_limit_mb). DGX Spark: 90% of 128 GB.
• Fit = total ≤ usable memory. headroomGB is usable memory minus total (negative means short by that much).
• Minimum GPUs (minCount): the smallest of 1, 2, 4 or 8 identical GPUs where each holds (weights + KV cache) ÷ n plus overhead, as with tensor parallelism in vLLM or SGLang. null means more than 8 GPUs (multi-node) or a device that is not normally combined (Macs, DGX Spark).

Not included: training or fine-tuning memory (optimizer states, gradients), CPU or disk offload, speculative-decoding draft models, vision encoders' activations, and engine-specific pre-allocation (vLLM reserves gpu_memory_utilization of the card, 90% by default, and fills the rest with KV cache).

Pricing

This Actor uses pay-per-event pricing:

Event	Price
Estimate returned (one model × quantisation row)	$0.003
Run start (per GB of memory, default run uses 256 MB = one event)	$0.005

Example: 5 models × 3 quantisations = 15 rows = $0.045 + $0.005 = $0.05. Models that fail (gated without a token, not found) are not charged. Set Max total charge on the run and the Actor stops before it goes over.

Use with AI agents and the Apify MCP server

Coding agents and chat assistants are often asked "will this model run on my GPU?" This Actor gives them a grounded answer from the model's real config instead of a guess. Pay-per-event pricing and fast runs make it a good MCP tool for Claude, ChatGPT, Cursor, n8n or LangChain through the Apify MCP server.

Add only this Actor as a tool:

{
  "mcpServers": {
    "llm-vram-calculator": { "url": "https://mcp.apify.com?tools=punkjazz-labs/llm-vram-calculator" }
  }
}

Or run the server locally: npx -y @apify/actors-mcp-server --tools punkjazz-labs/llm-vram-calculator.

Example prompts:

• "Can I run Qwen3-30B-A3B on my RTX 4090 at 32k context? Which quantisation?"
• "How many H100s do I need for DeepSeek-V3 in FP8?"
• "What's the biggest Qwen3 model that fits on a 64 GB MacBook at Q4_K_M?"

API

curl -X POST "https://api.apify.com/v2/acts/punkjazz-labs~llm-vram-calculator/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" -H 'Content-Type: application/json' \
  -d '{"modelIds": ["Qwen/Qwen3-30B-A3B"], "quantizations": ["bf16", "q4_k_m"], "contextLength": 32768}'

Use cases

• Choosing a GPU, cloud instance or Mac for a model before you buy or rent
• Picking the quantisation and context length that fit the hardware you already have
• Comparing many models at once for a self-hosting shortlist
• Capacity planning for vLLM, SGLang, TGI, llama.cpp, Ollama or LM Studio deployments
• Giving an AI agent a reliable "will it fit?" tool

FAQ

How accurate is it? Weights are usually within a few percent of the real file size. Total memory depends on your inference engine and its settings, so treat the result as a planning estimate and keep some headroom.

Why does my Llama or Gemma model fail? Those repos are gated. Accept the licence on the model page on Hugging Face, create a read token at huggingface.co/settings/tokens and paste it into Hugging Face token. The token is a secret input: it is encrypted, only sent to huggingface.co and never logged or written to the results.

Does it download the model? No. It reads only the API metadata and config.json, a few kilobytes per model.

Why is a MoE model's memory based on total parameters? All experts must be loaded, even though each token only uses a few of them. activeParamsB tells you the compute per token, which drives speed, not memory.

Does it work for GGUF repos? Yes. It uses the GGUF parameter count and, if the repo has no config.json, the base model's config.

Can I add my own GPU? Yes. Add "Name=GB" to Target GPUs / devices, for example "RTX A6000=48".

Does it cover training? No. Training and fine-tuning need memory for gradients and optimizer states, often 4 to 8 times more than inference.

Can it rank models by which fit my GPU? Yes. Use a search query plus targetDevices with your one GPU, then sort the results by totalVramGB or filter on devices[0].fits.

Related Actors

• UAE FTA Tax News Monitor: VAT and Corporate Tax updates for UAE businesses.
• Dubai eSupply Tenders Scraper: Dubai Government RFQs and RFPs.

Notes

• Estimates only, not measurements. Not affiliated with Hugging Face, NVIDIA, AMD or Apple.
• Questions, missing GPUs or wrong numbers for a model: open an issue in the Issues tab on this page.