Model family · 7 sizes
Mistral: which size runs locally?
Mistral comes in 7 sizes, from 7B to 46.7B. Its strongest tracked here, Mistral Small 3 24B, scores an LMArena Elo of 1357. Here is each size with its Q4_K_M weight, the memory it needs, and the hardware that runs it.
- Sizes
- 7
- Smallest
- 7B
- Largest
- 46.7B
- Runs from
- 16GB
The Mistral lineup
- Mistral 7B7B · ~4.37 GB Q4_K_M · needs ~8 GB · Elo 1149
- Mistral Nemo 12B12.2B · ~6.96 GB Q4_K_M · needs ~9 GB
- Mistral Small 3 24B24B · ~14.33 GB Q4_K_M · needs ~16 GB · Elo 1357
- Mistral Small 3.1 24B24B · ~13.35 GB Q4_K_M
- Magistral Small24B · ~14 GB Q4_K_M
- Devstral Small24B · ~13.35 GB Q4_K_M
- Mixtral 8x7B46.7B MoE · ~26.49 GB Q4_K_M · needs ~30 GB
"Needs" is the sourced minimum memory for Q4_K_M with a small context. Larger context needs more.
Which Mistral fits your memory
No Mistral size fits 8GB; even Mistral 7B needs more.
Largest that fits: Mistral Nemo 12B (12.2B), best case on Nvidia GeForce RTX 4080 (16GB).
Largest that fits: Devstral Small (24B), best case on Nvidia GeForce RTX 4090 (24GB).
Largest that fits: Mixtral 8x7B (46.7B), best case on Nvidia GeForce RTX 5090 (32GB). Comfortable up to Devstral Small (24B).
Largest that fits: Mixtral 8x7B (46.7B), best case on Apple M5 Pro (48GB). Comfortable up to Devstral Small (24B).
Largest that fits: Mixtral 8x7B (46.7B), best case on Apple M4 Max (64GB).
Largest that fits: Mixtral 8x7B (46.7B), best case on Apple M5 Max (128GB).
Largest that fits: Mixtral 8x7B (46.7B), best case on Apple M3 Ultra (256GB).
Best case means the most capable device at that size (usually a discrete GPU). A Mac at the same size sits roughly one rung lower; see the per-size breakdown on each memory budget page.
FAQ
Which Mistral size should I run locally?
Pick the largest size your memory allows. On 16GB (best case) up to Mistral Nemo 12B; On 24GB (best case) up to Devstral Small; On 32GB (best case) up to Mixtral 8x7B; On 48GB (best case) up to Mixtral 8x7B; On 64GB (best case) up to Mixtral 8x7B; On 128GB (best case) up to Mixtral 8x7B; On 256GB (best case) up to Mixtral 8x7B. Smaller sizes run faster and leave headroom for context.
What is the smallest Mistral model?
Mistral 7B at 7B parameters, about 4.37 GB on disk at Q4_K_M and roughly 8 GB of memory to run. It is the one to use on phones and 8 GB machines.
What is the largest Mistral model and what does it need?
Mixtral 8x7B at 46.7B (mixture of experts), about 26.49 GB at Q4_K_M and roughly 30 GB of memory. It fits a high-memory desktop GPU or Mac.
Understand the numbers
Short guides to the ideas behind Mistral's memory and quant figures.
Sources
- docs.mistral.ai
- huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF
- huggingface.co/bartowski/Mistral-Nemo-Instruct-2407-GGUF
- huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF
- huggingface.co/bartowski/mistralai_Mistral-Small-3.2-24B-Instruct-2506-GGUF
- huggingface.co/mistralai/Magistral-Small-2506
- huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
- huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503
- lmarena.ai
- ollama.com/library/mistral
- ollama.com/library/mistral-nemo
- ollama.com/library/mistral-nemo/tags
- ollama.com/library/mistral-small3.1
- ollama.com/library/mistral-small3.2
- ollama.com/library/mistral-small3.2/tags
- ollama.com/library/mistral/tags
Memory figures are estimates at Q4_K_M. See methodology.