Guide · Memory budgets
Is 6 GB of VRAM enough for a local LLM?
Yes, within limits: 38 of the 149 open models tracked here fit a 6 GB card at Q4_K_M with 4K context, topping out around 5.1B parameters. Here is exactly what fits, where it breaks, and what one step up to 8 GB buys.
A 6 GB card like the Nvidia GeForce RTX 2060 (6GB) keeps about 5 GB usable for weights. That runs the 5.1B-and-under class well at Q4_K_M. The 7B class needs CPU offload and gets slow. 8 GB is the first tier where 7B fits fully on the card.
What fits on a 6GB card
A model fits when its measured Q4_K_M file plus a 4K-context KV cache and runtime overhead stays inside the ~5 GB a 6 GB card keeps usable (the card also holds the runtime and your display buffer; the convention is documented in the methodology). The strongest models that clear that bar:
- Gemma 3 4B ~3.8 GB
- Llama 3.2 3B ~3.2 GB
- SmolLM2 1.7B ~2.2 GB
- Llama 3.2 1B ~1.8 GB
- Gemma 3 1B ~1.8 GB
- Qwen2.5 0.5B ~1.5 GB
- Qwen2.5 1.5B ~2.2 GB
- Qwen2.5 3B ~3.3 GB
Where it breaks: 7B and context
The popular 7B to 8B class lands just past the budget: Llama 3.1 8B alone is 4.92 GB on disk at Q4_K_M before the KV cache. Runtimes will still run it by spilling layers to CPU RAM, but every offloaded layer moves at system-memory speed and token rates drop sharply.
Context length is the second squeeze. The KV cache grows with every token of context and shares the card with the weights. The same model, three context settings:
- 4K context ~3.8 GB (0.5 GB of it KV cache)
- 16K context ~5.2 GB (1.9 GB of it KV cache)
- 32K context ~7.1 GB (3.8 GB of it KV cache)
What 8GB adds
One tier up, an 8 GB card like the Nvidia GeForce RTX 3060 Ti (8GB) keeps ~7 GB usable and the fit count rises from 38 to 54 models. The difference is almost entirely the 7B class fitting fully on the card. For the tier-by-tier view from there, see which LLMs fit in 16, 24 or 32 GB, check any model against any machine on the VRAM calculator, or start from your exact device.
FAQ
Is 6GB of VRAM enough to run a local LLM?
Yes, for the small end of the catalog: 38 of the 149 tracked open models fit a 6GB card at Q4_K_M with 4K context, topping out around 5.1B parameters (Gemma 4 E2B at 3.11 GB on disk). The 7B-and-up class does not fit without CPU offload, which is much slower.
Can I run a 7B model on 6GB VRAM?
Not fully on the card. A typical 7B to 8B model needs 4.92 GB on disk at Q4_K_M plus KV cache and overhead, which lands past the ~5 GB a card like the Nvidia GeForce RTX 2060 (6GB) keeps usable. Runtimes will offload the spill to CPU RAM, but token speed drops sharply.
How much better is 8GB of VRAM?
Meaningfully: the fit count rises from 38 models at 6GB to 54 at 8GB, and the 7B class starts fitting fully on the card at Q4_K_M.
Does longer context change the answer?
Yes. The KV cache grows with context length and competes with weights for the same VRAM. Gemma 3 4B needs ~3.8 GB total at 4K context but ~7.1 GB at 32K, so a card that fits it at 4K can run out of room purely from context.