Skip to content
localmodel.run

Text model · Mistral

Mistral 7B: RAM and VRAM requirements

Mistral 7B needs about 5.8 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~4.37 GB to download; KV cache and overhead add the rest), or about 9.1 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 3060 (12GB).

Mistral family · 7B params · released May 2024 · 26.1M Ollama pulls · LMArena Elo 1149.

Shopping for hardware? See what runs Mistral 7B →

License Apache-2.0 · Commercial OK ↓ 5.1M/mo ♥ 2.7K on HuggingFace
Q4_K_M GGUF
4.37 GB
Q8_0 GGUF
7.7 GB
Memory @ Q4 (4k)
~5.8 GB
Context
32 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Mistral 7B runs on 33 of 40 tracked devices at Q4_K_M.

33 run well 0 tight fit 7 too small
Fit check Q4_K_M
Yes, it runs fast
needs 5.8 GB usable 10.5 GB
See the full breakdown
$ollama run mistral:7b

Memory breakdown

Weights (Q4_K_M)4.37 GB
+
KV cache (4k)0.6 GB
+
Overhead0.8 GB
=
Total5.8 GB

How context length changes it

4k context ~5.8 GB 32k context ~10.3 GB 128k context ~25.5 GB

Longer context grows the KV cache quickly: Mistral 7B needs ~5.8 GB at 4k but ~25.5 GB at 128k, which can push it past a device that fits it at a short context.

Speed drops too: every token re-reads the KV cache, so at 128k context Mistral 7B generates at roughly ~18% of its short-context speed. How this is estimated.

Benchmark scores

LMArena · chat (elo)

#14 of 16 ranked · Human-preference Elo from LMArena's blind head-to-head votes.

1149

Sourced third-party benchmarks for the full-precision Mistral 7B, not a specific quant. See the full leaderboard.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 2.9 GB est
Q3_K_M 3.4 GB est
Q4_K_M (default) 4.37 GB
Q5_K_M 5 GB est
Q6_K 5.7 GB est
Q8_0 7.7 GB
FP16 14 GB est

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

Ollama
$ ollama run mistral:7b
llama.cpp
$ llama-cli -hf bartowski/Mistral-7B-Instruct-v0.3-GGUF:Q4_K_M
LM Studio
$ lms get bartowski/Mistral-7B-Instruct-v0.3-GGUF

Which devices can run Mistral 7B?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Gemma 4 E2B 5.1B Next size up Llama 3.1 8B 8B

Similar models

Head-to-head

FAQ

How much VRAM or RAM does Mistral 7B need?

At Q4_K_M, Mistral 7B needs about 5.8 GB (weights ~4.37 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~9.1 GB.

What is the Q4_K_M GGUF file size of Mistral 7B?

The Q4_K_M GGUF file is about 4.37 GB to download, and the Q8_0 GGUF is about 7.7 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~5.8 GB of memory at Q4_K_M.

Can Mistral 7B run on a laptop?

Yes, Mistral 7B fits on a 16 GB machine at Q4_K_M and runs on Apple Silicon or a 12 GB+ GPU comfortably.

Can I use Mistral 7B commercially?

Yes. Mistral 7B is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Mistral 7B's memory and quant figures.

Latest is v0.3 (released May 2024), adds function calling and extended vocabulary. Ollama default tag shows 4.4GB; bartowski HF repo gives precise 4.37GB Q4_K_M and 7.70GB Q8_0 for v0.3.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.