GPU VRAM calculator
GPU VRAM calculator: what a local AI model needs, and how many users it serves
An open AI model needs GPU memory for three things: its weights, a cache for each person's conversation, and the serving engine. Pick any model on Hugging Face and how many people will use it. You get the GPU memory, the cards that hold it, and the server around them, worked out from the model's own published files.
Context per person: more context lets the model work with longer conversations and more files, and needs more GPU memory for every person. As a guide: 8K for short chat, 32K for chat over documents, 64K for AI agents, whose instructions, tools and files all count.
80 users means about 16 people working at the same moment. That share is a planning assumption; if you know yours, choose the total that matches it.
Open models from the main publishers
86 models for 80 users. Choose one for its full result below the table, or click a column to sort. List from 2026-11-05.
| What you need | ||||||
|---|---|---|---|---|---|---|
| Alibaba (Qwen)China | Qwen3-4B | 4.0B | 8.0 GB | 101 GB | 7,241,469 | 1 × H200 NVL |
| Alibaba (Qwen)China | Qwen3-8B | 8.2B | 16.4 GB | 109 GB | 7,196,350 | 1 × H200 NVL |
| Alibaba (Qwen)China | Qwen3-14B | 15B | 29.5 GB | 131 GB | 12,937,511 | 1 × H200 NVL |
| Alibaba (Qwen)China | Qwen3-Coder-Next | 80B | 159 GB | 179 GB | 426,646 | 2 × RTX PRO 6000 Blackwell |
| Alibaba (Qwen)China | Qwen3-VL-8B-Instruct | 8.8B | 17.5 GB | 164 GB | 7,011,685 | 1 × B200 |
| Alibaba (Qwen)China | Qwen3.5-4B | 4.7B | 9.3 GB | 43.8 GB | 5,751,895 | 1 × L40S |
| Alibaba (Qwen)China | Qwen3.5-9B | 9.7B | 19.3 GB | 53.3 GB | 6,316,952 | 1 × A100 |
| Alibaba (Qwen)China | Qwen3.6-35B-A3B | 36B | 71.9 GB | 91.3 GB | 4,409,406 | 1 × H200 NVL |
| Alibaba (Qwen)China | Qwen3.8-2.4T-A95B | 2.4T | 4892 GB | 4743 GB | 26,684 | A cluster of about 28 × B200 across 4 servers |
| Alibaba (Qwen)China | Qwen3.8-27B | 28B | 55.6 GB | 120 GB | 8,037,597 | 1 × H200 NVL |
| Alibaba (Qwen)China | Qwen3.8-Flash-Next | 180B | 360 GB | 369 GB | 3,743,202 | 3 × H200 NVL |
| DeepSeekChina | DeepSeek-R1 | 684B | 689 GB | 3680 GB | 1,657,320 | 3 servers, each with 8 × B200 |
| DeepSeekChina | DeepSeek-V3 | 684B | 689 GB | 3680 GB | 1,594,819 | 3 servers, each with 8 × B200 |
| DeepSeekChina | DeepSeek-V3.2 | 685B | 689 GB | 3682 GB | 7,140,331 | 3 servers, each with 8 × B200 |
| DeepSeekChina | DeepSeek-V4-Flash-0731 | 304B | 167 GB | 248 GB | 2,874,001 | 2 × H200 NVL |
| DeepSeekChina | DeepSeek-V4.1-Flash | 763B | 510 GB | 568 GB | 1,826,303 | 4 × B200 |
| DeepSeekChina | DeepSeek-V4.1-Flash-1017 | 763B | 510 GB | 568 GB | 243,508 | 4 × B200 |
| GoogleUnited States | diffusiongemma-26B-A4B-it | 26B | 51.6 GB | 292 GB | 626,317 | 2 × B200 |
| GoogleUnited States | gemma-3-4b-itgated | 4.3B | 8.6 GB | ≥ 11.1 GB | 982,335 | At least 1 × GeForce RTX 5070 |
| GoogleUnited States | gemma-4-26B-A4B-it | 26B | 51.6 GB | 292 GB | 5,707,846 | 2 × B200 |
| GoogleUnited States | gemma-4-31B-it | 31B | 62.5 GB | 1023 GB | 7,684,597 | 6 × B200 |
| GoogleUnited States | medgemma-4b-itgated | 4.3B | 8.6 GB | ≥ 11.1 GB | 418,309 | At least 1 × GeForce RTX 5070 |
| IBMUnited States | granite-3.1-3b-a800m-instruct | 3.3B | 6.6 GB | 73.2 GB | 110,596 | 1 × A100 |
| IBMUnited States | granite-3.3-2b-instruct | 2.5B | 5.1 GB | 87.8 GB | 233,241 | 1 × RTX PRO 6000 Blackwell |
| IBMUnited States | granite-4.1-3b | 3.4B | 6.8 GB | 89.4 GB | 526,336 | 1 × RTX PRO 6000 Blackwell |
| IBMUnited States | granite-4.1-8b | 8.8B | 17.6 GB | 180 GB | 98,951 | 2 × RTX PRO 6000 Blackwell |
| IBMUnited States | granite-vision-4.1-4b | 4.0B | 8.0 GB | 90.6 GB | 117,229 | 1 × RTX PRO 6000 Blackwell |
| Liquid AIUnited States | d1-2B | 1.9B | 3.8 GB | 54.6 GB | 7,716 | 1 × A100 |
| Liquid AIUnited States | d1-3B | 3.1B | 6.2 GB | 38.9 GB | 37,515 | 1 × L40S |
| Liquid AIUnited States | LFM2.5-1.2B-Instruct | 1.2B | 2.3 GB | 37.2 GB | 119,291 | 1 × L40S |
| Liquid AIUnited States | LFM2.5-2.6B | 2.7B | 5.4 GB | 68.1 GB | 72,138 | 1 × A100 |
| Liquid AIUnited States | LFM2.5-24B-A2B | 24B | 47.9 GB | 129 GB | 35,149 | 1 × H200 NVL |
| Liquid AIUnited States | LFM2.5-24B-A2B-FP8 | 24B | 24.7 GB | 107 GB | 35,615 | 1 × H200 NVL |
| Liquid AIUnited States | LFM3-1.7B | 1.7B | 3.5 GB | 54.3 GB | 60,681 | 1 × A100 |
| Liquid AIUnited States | LFM3-VL-2B | 1.9B | 3.8 GB | 54.6 GB | 13,401 | 1 × A100 |
| MetaUnited States | Llama-3.1-8Bgated | 8.0B | 16.1 GB | 147 GB | 530,990 | 1 × B200 |
| MetaUnited States | Llama-3.1-8B-Instructgated | 8.0B | 16.1 GB | 147 GB | 4,330,207 | 1 × B200 |
| MetaUnited States | Llama-3.2-1Bgated | 1.2B | 2.5 GB | ≥ 5.3 GB | 476,702 | At least 1 × GeForce RTX 5070 |
| MetaUnited States | Llama-3.2-1B-Instructgated | 1.2B | 2.5 GB | ≥ 5.3 GB | 1,588,194 | At least 1 × GeForce RTX 5070 |
| MetaUnited States | Llama-3.2-3B-Instructgated | 3.2B | 6.4 GB | ≥ 9.1 GB | 948,230 | At least 1 × GeForce RTX 5070 |
| MetaUnited States | Meta-Llama-3-8B-Instructgated | 8.0B | 16.1 GB | 34.2 GB | 348,997 | 1 × L40S |
| MetaUnited States | Muse-Glimmer-30B | 30B | 59.6 GB | 112 GB | 186,321 | 1 × H200 NVL |
| MetaUnited States | Muse-Spark-Open-816B-A42Bgated | 822B | 1644 GB | ≥ 1565 GB | 11,272 | At least A cluster of about 10 × B200 across 2 servers |
| MetaUnited States | Muse-Spark-Open-816B-A42B-NVFP4gated | 484B | 533 GB | ≥ 510 GB | 15,518 | At least 3 × B200 |
| MicrosoftUnited States | Mage-VL-Cowork | 4.7B | 9.5 GB | 156 GB | 2,555 | 1 × B200 |
| MicrosoftUnited States | Phi-3-mini-4k-instruct | 3.8B | 7.6 GB | 34.2 GB | 322,406 | 1 × L40S |
| MicrosoftUnited States | Phi-3.5-mini-instruct | 3.8B | 7.6 GB | 395 GB | 384,018 | 3 × H200 NVL |
| MicrosoftUnited States | Phi-3.5-vision-instruct | 4.1B | 8.3 GB | 395 GB | 324,087 | 3 × H200 NVL |
| MicrosoftUnited States | phi-4 | 15B | 29.3 GB | 80.8 GB | 306,815 | 1 × RTX PRO 6000 Blackwell |
| MicrosoftUnited States | Phi-4-mini-instruct | 3.8B | 7.7 GB | 139 GB | 336,440 | 1 × B200 |
| MicrosoftUnited States | UserLM-2.0 | 8.0B | 32.1 GB | 162 GB | 496 | 1 × B200 |
| MiniMaxChina | MiniMax-M2 | 229B | 230 GB | 470 GB | 105,509 | 3 × B200 |
| MiniMaxChina | MiniMax-M2.5 | 229B | 230 GB | 470 GB | 1,270,024 | 3 × B200 |
| MiniMaxChina | MiniMax-M2.7 | 229B | 230 GB | 470 GB | 322,443 | 3 × B200 |
| MiniMaxChina | MiniMax-M3 | 427B | 854 GB | 935 GB | 165,850 | 6 × B200 |
| MiniMaxChina | MiniMax-M3-MXFP8 | 427B | 444 GB | 545 GB | 42,809 | 4 × B200 |
| MiniMaxChina | MiniMax-M3.1-Flash | 220B | 250 GB | 361 GB | 34,815 | 3 × H200 NVL |
| Mistral AIFrance | Ministral-3-14B-Instruct-2512 | 14B | 15.7 GB | 178 GB | 160,533 | 2 × RTX PRO 6000 Blackwell |
| Mistral AIFrance | Ministral-3-14B-Reasoning-2512 | 14B | 27.9 GB | 190 GB | 307,870 | 2 × H200 NVL |
| Mistral AIFrance | Mistral-7B-Instruct-v0.2 | 7.2B | 14.5 GB | 80.7 GB | 1,626,284 | 1 × RTX PRO 6000 Blackwell |
| Mistral AIFrance | Mistral-7B-Instruct-v0.3 | 7.2B | 14.5 GB | 80.8 GB | 1,685,649 | 1 × RTX PRO 6000 Blackwell |
| Mistral AIFrance | Mistral-7B-v0.1 | 7.2B | 14.5 GB | 80.7 GB | 470,345 | 1 × RTX PRO 6000 Blackwell |
| Mistral AIFrance | Mixtral-8x7B-Instruct-v0.1 | 47B | 93.4 GB | 156 GB | 290,215 | 1 × B200 |
| Moonshot AIChina | Kimi-K2-Instruct | 1.0T | 1029 GB | 7592 GB | 306,336 | 6 servers, each with 8 × B200 |
| Moonshot AIChina | Kimi-K2.5 | 1.0T | 595 GB | 3413 GB | 394,498 | 3 servers, each with 8 × B200 |
| Moonshot AIChina | Kimi-K2.6 | 1.0T | 595 GB | 3413 GB | 240,282 | 3 servers, each with 8 × B200 |
| Moonshot AIChina | Kimi-K2.7-Code | 1.0T | 595 GB | 3413 GB | 83,330 | 3 servers, each with 8 × B200 |
| Moonshot AIChina | Kimi-K3 | 2.8T | 1561 GB | 4067 GB | 777,904 | A cluster of about 24 × B200 across 3 servers |
| Moonshot AIChina | Kimi-K3.1-Pro | 2.8T | 1561 GB | 4067 GB | 11,658 | A cluster of about 24 × B200 across 3 servers |
| NVIDIAUnited States | Cosmos-Reason2-2Bgated | 2.4B | 4.9 GB | 120 GB | 601,472 | 1 × H200 NVL |
| NVIDIAUnited States | LocateAnything-3B | 3.8B | 7.7 GB | 28.3 GB | 244,364 | 1 × GeForce RTX 5090 |
| NVIDIAUnited States | NVIDIA-Nemotron-3-Nano-4B-BF16 | 4.0B | 7.9 GB | 179 GB | 1,879,446 | 2 × RTX PRO 6000 Blackwell |
| NVIDIAUnited States | NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 32B | 63.2 GB | 115 GB | 1,641,135 | 1 × H200 NVL |
| NVIDIAUnited States | NVIDIA-Nemotron-3-Super-120B-A12B-BF16 | 124B | 247 GB | 326 GB | 1,081,075 | 2 × B200 |
| NVIDIAUnited States | NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | 561B | 1121 GB | ≥ 1068 GB | 1,221,229 | At least 7 × B200 |
| NVIDIAUnited States | NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 | 32B | 65.8 GB | 118 GB | 1,119,938 | 1 × H200 NVL |
| NVIDIAUnited States | NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 | 66B | 75.0 GB | ≥ 74.2 GB | 36,887 | At least 1 × A100 |
| NVIDIAUnited States | NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 | 4.0B | 8.0 GB | 299 GB | 11,694 | 2 × B200 |
| NVIDIAUnited States | NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 | 2.3B | 3.2 GB | 294 GB | 3,973 | 2 × B200 |
| OpenAIUnited States | gpt-oss-20b | 21B | 13.8 GB | 64.1 GB | 5,941,800 | 1 × A100 |
| OpenAIUnited States | gpt-oss-120b | 117B | 65.2 GB | 137 GB | 2,428,828 | 1 × B200 |
| Z.aiChina | GLM-4.5-Air | 110B | 221 GB | 397 GB | 1,272,138 | 3 × H200 NVL |
| Z.aiChina | GLM-4.7-Flash | 31B | 62.4 GB | 437 GB | 981,743 | 3 × B200 |
| Z.aiChina | GLM-5 | 754B | 1508 GB | 2684 GB | 1,491,776 | A cluster of about 16 × B200 across 2 servers |
| Z.aiChina | GLM-5.3 | 753B | 756 GB | 9510 GB | 2,255,382 | 8 servers, each with 7 × B200 |
| Z.aiChina | GLM-5.3-Flash | 321B | 328 GB | 1035 GB | 6,277,088 | 7 × B200 |
How the figures are worked out
GPU memory = weights + conversation cache + engine. Weights are the repository's own top-level files, or its parameter count when you choose a smaller precision (about 1 byte per parameter at 8-bit, 0.62 at 4-bit). The cache is context × people × the bytes per token that follow from the model's config.json: cached layers × key-value heads × head size × 2 × 2 bytes. The engine adds 3 GB under vLLM and 1 GB under llama.cpp. We keep 5% of each card free. A server is limited to 2 desktop cards, 4 workstation cards, or 8 data-center cards or RTX PRO 6000s, and each extra server runs its own copy of the model. The answer is the fewest servers, then the fewest cards.
Checked against a real server. gpt-oss-20b at 32K context used 17.2 GB on our own RTX A5000 under vLLM. This page says 17.6 GB. We use the same arithmetic in our product.
What it does not tell you. Speed. Memory says how many people fit, not how quickly they get answers. In our own test, four people sending an 8,000-token conversation together on one 24 GB RTX A5000 under llama.cpp, an engine that answered them one after another, all had answers within 45 seconds. Engines that batch requests behave differently, and we have not measured large groups. Estimates also assume each person uses the full context all the time, so they are the safe upper figure, and a real team usually needs less.
Larger than one server. When a model needs more memory than one server's cards hold, NeueCode can run it as one copy split across several servers on your own network. Over an ordinary link it answers more slowly than on one server; over a fast private link of 20 Gbit/s the servers share every layer of the model.
Where the data comes from. Your browser asks Hugging Face directly, so Hugging Face sees those requests. NeueCode is not affiliated with Hugging Face or NVIDIA. The maker's country is shown for the publishers we list, and it says nothing about a model's quality or safety. Check each model's license before you use it.
Use and cite it. It is free to use. Cite it as: NeueCode, GPU VRAM Calculator for AI Models, https://neuecode.com/ai-model-hardware, data from Hugging Face, checked 2026-11-05. The figures for every listed model are at model-hardware.json.
Questions
How much GPU memory does an AI model need?
Three parts: the model's weights, a cache for each person's conversation, and the serving engine itself. A 14 GB download such as gpt-oss-20b needs about 18 GB for one person at 32K context, and about 3 GB more for each extra person at 64K. Choose a model above to see its figure.
How many users can one GPU serve?
The page gives the limit memory sets, for any model on any card. Not everyone works at once: published sizing guides plan chat at 1 person in 10 to 20. We plan chat at 1 in 15, and AI agents at 1 in 5, because an agent run holds the model much longer than a chat answer. Speed can set a lower limit, so load-test before you buy.
What does context per person mean?
Context is how much text the model can hold at once for one person: their conversation, the files it reads, and for AI agents its instructions and tools. It is counted in tokens, and a token is roughly three quarters of an English word, so 64K is about 48,000 words. Each person at once needs their own context, so more context needs more GPU memory. As a guide: 8K for short chat, 32K for chat over documents, 64K for AI agents.
Can I run a model across several GPUs?
Inside one server, some serving engines can split a model across its cards, and the page assumes that when it names several cards in a server. Each extra server runs its own copy of the model, so it has to hold the weights again. Some engines and products run a model on one card only, so check yours first.
Do I need a Hugging Face account or key?
No. The page reads public data, and we have added what it can for the gated models on our list. A gated model we have no access to shows its size only, and the figure says "at least".
What does 4-bit or 8-bit mean?
Quantization stores each weight in fewer bits, so the model takes less memory, usually with some loss of quality. A model published already quantized does not shrink again here.
Running a model on your own servers?
NeueCode is private autonomous intelligence for enterprise: it takes on your organization's tasks on your own GPUs, with nothing leaving your network, a person approving every action, and all of it on record. It runs each model on one card, so read the single-card figures above.
What you need to run NeueCode