Local AI GuideHow to use it
- Type the model's size in billions of parameters and the bits per weight of the file you will use.
- For the KV cache, type the layers, KV heads and head size from the model's configuration (the values shown are an example, not a particular model), and the context you want.
- Type your VRAM to see whether it fits. The result also shows Ollama's documented default context for that much VRAM.
What it is
An estimate from arithmetic: weights plus KV cache plus your margin. It cannot know how a program lays out memory. Check with the real program: in Ollama, ollama ps shows the size of a loaded model and how much of it runs on the GPU.
Where do I find layers and KV heads?
In the model's configuration file or card, on the page where you download it.
Does it work for a CPU-only computer?
The arithmetic is the same for system memory, but speed will be slower.
This is an independent guide. It is not connected with Ollama, the llama.cpp project or any company named here. This field changes quickly: statements about Ollama and llama.cpp were read on 14 November 2026 from the pages named under Sources. The memory numbers are estimates from arithmetic. We did not run any model for these pages.