Local AI GuideHow much memory a local model needs
Two things fill memory: the model's weights and the context you give it.
Read on 14 November 2026. This field changes fast: check the sources.
Weights
The weights are the model's numbers. Their size is the number of parameters times the bits used for each one, divided by 8 for bytes. A 7 billion parameter model stored with 16 bits needs about 13 GiB, with 8 bits about 6.5 GiB, and with 4 bits about 3.3 GiB:
| Parameters | At 16 bits | At 8 bits | At 4 bits |
|---|---|---|---|
| 1 billion | 1.9 GiB | 0.9 GiB | 0.5 GiB |
| 3 billion | 5.6 GiB | 2.8 GiB | 1.4 GiB |
| 7 billion | 13.0 GiB | 6.5 GiB | 3.3 GiB |
| 8 billion | 14.9 GiB | 7.5 GiB | 3.7 GiB |
| 13 billion | 24.2 GiB | 12.1 GiB | 6.1 GiB |
| 32 billion | 59.6 GiB | 29.8 GiB | 14.9 GiB |
| 70 billion | 130.4 GiB | 65.2 GiB | 32.6 GiB |
Compressing weights to fewer bits is called quantization (see its page). Real files differ a little because some parts are kept at higher precision.
The KV cache
While the model reads and writes text it keeps keys and values for every token of the context. Their size is 2 (keys and values) × layers × KV heads × head size × context tokens × bytes per value. For an example model with 32 layers, 8 KV heads and a head size of 128, using 16-bit values:
| Context (tokens) | KV cache (example model, 16-bit values) |
|---|---|
| 4,096 | 0.50 GiB |
| 8,192 | 1.00 GiB |
| 32,768 | 4.00 GiB |
| 131,072 | 16.00 GiB |
These are example architecture numbers, not any particular model: read the model's own configuration. The cache grows in proportion to the context, which is why a long context can need more memory than the weights.
Put them together
Total memory is about the weights plus the KV cache plus a margin for buffers. The estimator on the home page adds a margin you choose (for example 20 percent) and compares the total with your VRAM.
What Ollama says about context
Ollama's documentation says that context length is the maximum number of tokens the model has access to in memory, and that its defaults depend on VRAM: 4k below 24 GiB, 32k from 24 to 48 GiB and 256k from 48 GiB. It also says that tasks that need a large context, such as web search, agents and coding tools, should be set to at least 64,000 tokens, and that a larger context length increases the memory a model needs.
Why does my model not fit although the file is smaller than my VRAM?
The context (KV cache) and buffers need memory too.
Is system RAM the same as VRAM?
No. Models that do not fit in VRAM can be split between GPU and CPU, which is slower: see the Ollama settings page.
Sources
- Own arithmetic (checked against Python in the tests of this section)
- Ollama documentation: Context length (docs.ollama.com/context-length), read 14 November 2026
Read on 14 November 2026.
This is an independent guide. It is not connected with Ollama, the llama.cpp project or any company named here. This field changes quickly: statements about Ollama and llama.cpp were read on 14 November 2026 from the pages named under Sources. The memory numbers are estimates from arithmetic. We did not run any model for these pages.