GPU VRAM calculator

GPU VRAM calculator: what a local AI model needs, and how many users it serves

An open AI model needs GPU memory for three things: its weights, a cache for each person's conversation, and the serving engine. Pick any model on Hugging Face and how many people will use it. You get the GPU memory, the cards that hold it, and the server around them, worked out from the model's own published files.

Context per person: more context lets the model work with longer conversations and more files, and needs more GPU memory for every person. As a guide: 8K for short chat, 32K for chat over documents, 64K for AI agents, whose instructions, tools and files all count.

80 users means about 16 people working at the same moment. That share is a planning assumption; if you know yours, choose the total that matches it.

Open models from the main publishers

86 models for 80 users. Choose one for its full result below the table, or click a column to sort. List from 2026-11-05.

What you need
Alibaba (Qwen)ChinaQwen3-4B4.0B8.0 GB101 GB7,241,4691 × H200 NVL
Alibaba (Qwen)ChinaQwen3-8B8.2B16.4 GB109 GB7,196,3501 × H200 NVL
Alibaba (Qwen)ChinaQwen3-14B15B29.5 GB131 GB12,937,5111 × H200 NVL
Alibaba (Qwen)ChinaQwen3-Coder-Next80B159 GB179 GB426,6462 × RTX PRO 6000 Blackwell
Alibaba (Qwen)ChinaQwen3-VL-8B-Instruct8.8B17.5 GB164 GB7,011,6851 × B200
Alibaba (Qwen)ChinaQwen3.5-4B4.7B9.3 GB43.8 GB5,751,8951 × L40S
Alibaba (Qwen)ChinaQwen3.5-9B9.7B19.3 GB53.3 GB6,316,9521 × A100
Alibaba (Qwen)ChinaQwen3.6-35B-A3B36B71.9 GB91.3 GB4,409,4061 × H200 NVL
Alibaba (Qwen)ChinaQwen3.8-2.4T-A95B2.4T4892 GB4743 GB26,684A cluster of about 28 × B200 across 4 servers
Alibaba (Qwen)ChinaQwen3.8-27B28B55.6 GB120 GB8,037,5971 × H200 NVL
Alibaba (Qwen)ChinaQwen3.8-Flash-Next180B360 GB369 GB3,743,2023 × H200 NVL
DeepSeekChinaDeepSeek-R1684B689 GB3680 GB1,657,3203 servers, each with 8 × B200
DeepSeekChinaDeepSeek-V3684B689 GB3680 GB1,594,8193 servers, each with 8 × B200
DeepSeekChinaDeepSeek-V3.2685B689 GB3682 GB7,140,3313 servers, each with 8 × B200
DeepSeekChinaDeepSeek-V4-Flash-0731304B167 GB248 GB2,874,0012 × H200 NVL
DeepSeekChinaDeepSeek-V4.1-Flash763B510 GB568 GB1,826,3034 × B200
DeepSeekChinaDeepSeek-V4.1-Flash-1017763B510 GB568 GB243,5084 × B200
GoogleUnited Statesdiffusiongemma-26B-A4B-it26B51.6 GB292 GB626,3172 × B200
GoogleUnited Statesgemma-3-4b-itgated4.3B8.6 GB≥ 11.1 GB982,335At least 1 × GeForce RTX 5070
GoogleUnited Statesgemma-4-26B-A4B-it26B51.6 GB292 GB5,707,8462 × B200
GoogleUnited Statesgemma-4-31B-it31B62.5 GB1023 GB7,684,5976 × B200
GoogleUnited Statesmedgemma-4b-itgated4.3B8.6 GB≥ 11.1 GB418,309At least 1 × GeForce RTX 5070
IBMUnited Statesgranite-3.1-3b-a800m-instruct3.3B6.6 GB73.2 GB110,5961 × A100
IBMUnited Statesgranite-3.3-2b-instruct2.5B5.1 GB87.8 GB233,2411 × RTX PRO 6000 Blackwell
IBMUnited Statesgranite-4.1-3b3.4B6.8 GB89.4 GB526,3361 × RTX PRO 6000 Blackwell
IBMUnited Statesgranite-4.1-8b8.8B17.6 GB180 GB98,9512 × RTX PRO 6000 Blackwell
IBMUnited Statesgranite-vision-4.1-4b4.0B8.0 GB90.6 GB117,2291 × RTX PRO 6000 Blackwell
Liquid AIUnited Statesd1-2B1.9B3.8 GB54.6 GB7,7161 × A100
Liquid AIUnited Statesd1-3B3.1B6.2 GB38.9 GB37,5151 × L40S
Liquid AIUnited StatesLFM2.5-1.2B-Instruct1.2B2.3 GB37.2 GB119,2911 × L40S
Liquid AIUnited StatesLFM2.5-2.6B2.7B5.4 GB68.1 GB72,1381 × A100
Liquid AIUnited StatesLFM2.5-24B-A2B24B47.9 GB129 GB35,1491 × H200 NVL
Liquid AIUnited StatesLFM2.5-24B-A2B-FP824B24.7 GB107 GB35,6151 × H200 NVL
Liquid AIUnited StatesLFM3-1.7B1.7B3.5 GB54.3 GB60,6811 × A100
Liquid AIUnited StatesLFM3-VL-2B1.9B3.8 GB54.6 GB13,4011 × A100
MetaUnited StatesLlama-3.1-8Bgated8.0B16.1 GB147 GB530,9901 × B200
MetaUnited StatesLlama-3.1-8B-Instructgated8.0B16.1 GB147 GB4,330,2071 × B200
MetaUnited StatesLlama-3.2-1Bgated1.2B2.5 GB≥ 5.3 GB476,702At least 1 × GeForce RTX 5070
MetaUnited StatesLlama-3.2-1B-Instructgated1.2B2.5 GB≥ 5.3 GB1,588,194At least 1 × GeForce RTX 5070
MetaUnited StatesLlama-3.2-3B-Instructgated3.2B6.4 GB≥ 9.1 GB948,230At least 1 × GeForce RTX 5070
MetaUnited StatesMeta-Llama-3-8B-Instructgated8.0B16.1 GB34.2 GB348,9971 × L40S
MetaUnited StatesMuse-Glimmer-30B30B59.6 GB112 GB186,3211 × H200 NVL
MetaUnited StatesMuse-Spark-Open-816B-A42Bgated822B1644 GB≥ 1565 GB11,272At least A cluster of about 10 × B200 across 2 servers
MetaUnited StatesMuse-Spark-Open-816B-A42B-NVFP4gated484B533 GB≥ 510 GB15,518At least 3 × B200
MicrosoftUnited StatesMage-VL-Cowork4.7B9.5 GB156 GB2,5551 × B200
MicrosoftUnited StatesPhi-3-mini-4k-instruct3.8B7.6 GB34.2 GB322,4061 × L40S
MicrosoftUnited StatesPhi-3.5-mini-instruct3.8B7.6 GB395 GB384,0183 × H200 NVL
MicrosoftUnited StatesPhi-3.5-vision-instruct4.1B8.3 GB395 GB324,0873 × H200 NVL
MicrosoftUnited Statesphi-415B29.3 GB80.8 GB306,8151 × RTX PRO 6000 Blackwell
MicrosoftUnited StatesPhi-4-mini-instruct3.8B7.7 GB139 GB336,4401 × B200
MicrosoftUnited StatesUserLM-2.08.0B32.1 GB162 GB4961 × B200
MiniMaxChinaMiniMax-M2229B230 GB470 GB105,5093 × B200
MiniMaxChinaMiniMax-M2.5229B230 GB470 GB1,270,0243 × B200
MiniMaxChinaMiniMax-M2.7229B230 GB470 GB322,4433 × B200
MiniMaxChinaMiniMax-M3427B854 GB935 GB165,8506 × B200
MiniMaxChinaMiniMax-M3-MXFP8427B444 GB545 GB42,8094 × B200
MiniMaxChinaMiniMax-M3.1-Flash220B250 GB361 GB34,8153 × H200 NVL
Mistral AIFranceMinistral-3-14B-Instruct-251214B15.7 GB178 GB160,5332 × RTX PRO 6000 Blackwell
Mistral AIFranceMinistral-3-14B-Reasoning-251214B27.9 GB190 GB307,8702 × H200 NVL
Mistral AIFranceMistral-7B-Instruct-v0.27.2B14.5 GB80.7 GB1,626,2841 × RTX PRO 6000 Blackwell
Mistral AIFranceMistral-7B-Instruct-v0.37.2B14.5 GB80.8 GB1,685,6491 × RTX PRO 6000 Blackwell
Mistral AIFranceMistral-7B-v0.17.2B14.5 GB80.7 GB470,3451 × RTX PRO 6000 Blackwell
Mistral AIFranceMixtral-8x7B-Instruct-v0.147B93.4 GB156 GB290,2151 × B200
Moonshot AIChinaKimi-K2-Instruct1.0T1029 GB7592 GB306,3366 servers, each with 8 × B200
Moonshot AIChinaKimi-K2.51.0T595 GB3413 GB394,4983 servers, each with 8 × B200
Moonshot AIChinaKimi-K2.61.0T595 GB3413 GB240,2823 servers, each with 8 × B200
Moonshot AIChinaKimi-K2.7-Code1.0T595 GB3413 GB83,3303 servers, each with 8 × B200
Moonshot AIChinaKimi-K32.8T1561 GB4067 GB777,904A cluster of about 24 × B200 across 3 servers
Moonshot AIChinaKimi-K3.1-Pro2.8T1561 GB4067 GB11,658A cluster of about 24 × B200 across 3 servers
NVIDIAUnited StatesCosmos-Reason2-2Bgated2.4B4.9 GB120 GB601,4721 × H200 NVL
NVIDIAUnited StatesLocateAnything-3B3.8B7.7 GB28.3 GB244,3641 × GeForce RTX 5090
NVIDIAUnited StatesNVIDIA-Nemotron-3-Nano-4B-BF164.0B7.9 GB179 GB1,879,4462 × RTX PRO 6000 Blackwell
NVIDIAUnited StatesNVIDIA-Nemotron-3-Nano-30B-A3B-BF1632B63.2 GB115 GB1,641,1351 × H200 NVL
NVIDIAUnited StatesNVIDIA-Nemotron-3-Super-120B-A12B-BF16124B247 GB326 GB1,081,0752 × B200
NVIDIAUnited StatesNVIDIA-Nemotron-3-Ultra-550B-A55B-BF16561B1121 GB≥ 1068 GB1,221,229At least 7 × B200
NVIDIAUnited StatesNVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1632B65.8 GB118 GB1,119,9381 × H200 NVL
NVIDIAUnited StatesNVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP466B75.0 GB≥ 74.2 GB36,887At least 1 × A100
NVIDIAUnited StatesNVIDIA-Nemotron-Labs-Dory1.0-4B-BF164.0B8.0 GB299 GB11,6942 × B200
NVIDIAUnited StatesNVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP42.3B3.2 GB294 GB3,9732 × B200
OpenAIUnited Statesgpt-oss-20b21B13.8 GB64.1 GB5,941,8001 × A100
OpenAIUnited Statesgpt-oss-120b117B65.2 GB137 GB2,428,8281 × B200
Z.aiChinaGLM-4.5-Air110B221 GB397 GB1,272,1383 × H200 NVL
Z.aiChinaGLM-4.7-Flash31B62.4 GB437 GB981,7433 × B200
Z.aiChinaGLM-5754B1508 GB2684 GB1,491,776A cluster of about 16 × B200 across 2 servers
Z.aiChinaGLM-5.3753B756 GB9510 GB2,255,3828 servers, each with 7 × B200
Z.aiChinaGLM-5.3-Flash321B328 GB1035 GB6,277,0887 × B200

How the figures are worked out

GPU memory = weights + conversation cache + engine. Weights are the repository's own top-level files, or its parameter count when you choose a smaller precision (about 1 byte per parameter at 8-bit, 0.62 at 4-bit). The cache is context × people × the bytes per token that follow from the model's config.json: cached layers × key-value heads × head size × 2 × 2 bytes. The engine adds 3 GB under vLLM and 1 GB under llama.cpp. We keep 5% of each card free. A server is limited to 2 desktop cards, 4 workstation cards, or 8 data-center cards or RTX PRO 6000s, and each extra server runs its own copy of the model. The answer is the fewest servers, then the fewest cards.

Checked against a real server. gpt-oss-20b at 32K context used 17.2 GB on our own RTX A5000 under vLLM. This page says 17.6 GB. We use the same arithmetic in our product.

What it does not tell you. Speed. Memory says how many people fit, not how quickly they get answers. In our own test, four people sending an 8,000-token conversation together on one 24 GB RTX A5000 under llama.cpp, an engine that answered them one after another, all had answers within 45 seconds. Engines that batch requests behave differently, and we have not measured large groups. Estimates also assume each person uses the full context all the time, so they are the safe upper figure, and a real team usually needs less.

Larger than one server. When a model needs more memory than one server's cards hold, NeueCode can run it as one copy split across several servers on your own network. Over an ordinary link it answers more slowly than on one server; over a fast private link of 20 Gbit/s the servers share every layer of the model.

Where the data comes from. Your browser asks Hugging Face directly, so Hugging Face sees those requests. NeueCode is not affiliated with Hugging Face or NVIDIA. The maker's country is shown for the publishers we list, and it says nothing about a model's quality or safety. Check each model's license before you use it.

Use and cite it. It is free to use. Cite it as: NeueCode, GPU VRAM Calculator for AI Models, https://neuecode.com/ai-model-hardware, data from Hugging Face, checked 2026-11-05. The figures for every listed model are at model-hardware.json.

Questions

How much GPU memory does an AI model need?

Three parts: the model's weights, a cache for each person's conversation, and the serving engine itself. A 14 GB download such as gpt-oss-20b needs about 18 GB for one person at 32K context, and about 3 GB more for each extra person at 64K. Choose a model above to see its figure.

How many users can one GPU serve?

The page gives the limit memory sets, for any model on any card. Not everyone works at once: published sizing guides plan chat at 1 person in 10 to 20. We plan chat at 1 in 15, and AI agents at 1 in 5, because an agent run holds the model much longer than a chat answer. Speed can set a lower limit, so load-test before you buy.

What does context per person mean?

Context is how much text the model can hold at once for one person: their conversation, the files it reads, and for AI agents its instructions and tools. It is counted in tokens, and a token is roughly three quarters of an English word, so 64K is about 48,000 words. Each person at once needs their own context, so more context needs more GPU memory. As a guide: 8K for short chat, 32K for chat over documents, 64K for AI agents.

Can I run a model across several GPUs?

Inside one server, some serving engines can split a model across its cards, and the page assumes that when it names several cards in a server. Each extra server runs its own copy of the model, so it has to hold the weights again. Some engines and products run a model on one card only, so check yours first.

Do I need a Hugging Face account or key?

No. The page reads public data, and we have added what it can for the gated models on our list. A gated model we have no access to shows its size only, and the figure says "at least".

What does 4-bit or 8-bit mean?

Quantization stores each weight in fewer bits, so the model takes less memory, usually with some loss of quality. A model published already quantized does not shrink again here.

Running a model on your own servers?

NeueCode is private autonomous intelligence for enterprise: it takes on your organization's tasks on your own GPUs, with nothing leaving your network, a person approving every action, and all of it on record. It runs each model on one card, so read the single-card figures above.

What you need to run NeueCode