Skip to content
localmodel.run

Text model · Mistral

Mixtral 8x7B: RAM and VRAM requirements

Mixtral 8x7B needs about 28.9 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~26.49 GB to download; KV cache and overhead add the rest), or about 48.6 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 5090 (32GB).

Mistral family · 46.7B params (Mixture-of-Experts: activates only 12.9B of 46.7B params per token, so generation is faster than the total size suggests) · released Dec 2023 · 2.7M Ollama pulls.

Shopping for hardware? See what runs Mixtral 8x7B →

License Apache-2.0 · Commercial OK ↓ 896.4K/mo ♥ 4.7K on HuggingFace
Q4_K_M GGUF
26.49 GB
Q8_0 GGUF
46.22 GB
Memory @ Q4 (4k)
~28.9 GB
Context
32 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Mixtral 8x7B runs on 8 of 40 tracked devices at Q4_K_M.

5 run well 3 tight fit 32 too small
Fit check Q4_K_M
No, not enough memory
needs 28.9 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)26.49 GB
+
KV cache (4k)1.6 GB
+
Overhead0.8 GB
=
Total28.9 GB

How context length changes it

4k context ~28.9 GB 32k context ~40.4 GB 128k context ~79.8 GB

Longer context grows the KV cache, which for Mixtral 8x7B is sized by its 12.9B active params, not the full 46.7B. It needs ~28.9 GB at 4k and ~79.8 GB at 128k.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 19.6 GB est
Q3_K_M 22.8 GB est
Q4_K_M (default) 26.49 GB
Q5_K_M 33.3 GB est
Q6_K 38.3 GB est
Q8_0 46.22 GB
FP16 93.4 GB est

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

Ollama
$ ollama run mixtral:8x7b
llama.cpp
$ llama-cli -hf MaziyarPanahi/Mixtral-8x7B-Instruct-v0.1-GGUF:Q4_K_M
LM Studio
$ lms get MaziyarPanahi/Mixtral-8x7B-Instruct-v0.1-GGUF

Which devices can run Mixtral 8x7B?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Qwen3.6 35B-A3B 36B Next size up Llama-3.3-Nemotron-Super-49B-v1 49B

Similar models

Head-to-head

FAQ

How much VRAM or RAM does Mixtral 8x7B need?

At Q4_K_M, Mixtral 8x7B needs about 28.9 GB (weights ~26.49 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~48.6 GB.

What is the Q4_K_M GGUF file size of Mixtral 8x7B?

The Q4_K_M GGUF file is about 26.49 GB to download, and the Q8_0 GGUF is about 46.22 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~28.9 GB of memory at Q4_K_M.

Can Mixtral 8x7B run on a laptop?

Mixtral 8x7B is large; you need a high-memory Mac or multi-GPU setup at Q4_K_M.

Is Mixtral 8x7B cheaper to run because it is a MoE model?

It is faster, not lighter. Mixtral 8x7B activates only 12.9B of 46.7B params per token (so it runs quickly), but all experts must stay in memory, so it still needs memory for the full 46.7B.

Can I use Mixtral 8x7B commercially?

Yes. Mixtral 8x7B is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Mixtral 8x7B's memory and quant figures.

Mixtral 8x7B sparse MoE: 46.7B total, 12.9B active (2 of 8 experts per token). All experts must fit in memory. Q4_K_M and Q8_0 sizes from the MaziyarPanahi GGUF repo.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.