Skip to content
localmodel.run

Text model · Sarvam

S Sarvam-M 24B: RAM and VRAM requirements

Sarvam-M 24B needs about 16.3 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~14.3 GB to download; KV cache and overhead add the rest), or about 27.1 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 4090 (24GB).

Sarvam family · 24B params · released May 2025.

Shopping for hardware? See what runs Sarvam-M 24B →

License Apache-2.0 · Commercial OK ↓ 9.7K/mo ♥ 348 on HuggingFace
Q4_K_M GGUF
14.3 GB
Q8_0 GGUF
25.1 GB
Memory @ Q4 (4k)
~16.3 GB
Context
32 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Sarvam-M 24B runs on 13 of 40 tracked devices at Q4_K_M.

13 run well 0 tight fit 27 too small
Fit check Q4_K_M
No, not enough memory
needs 16.3 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)14.3 GB
+
KV cache (4k)1.2 GB
+
Overhead0.8 GB
=
Total16.3 GB

How context length changes it

4k context ~16.3 GB 32k context ~24.5 GB 128k context ~52.7 GB

Longer context grows the KV cache quickly: Sarvam-M 24B needs ~16.3 GB at 4k but ~52.7 GB at 128k, which can push it past a device that fits it at a short context.

Speed drops too: every token re-reads the KV cache, so at 128k context Sarvam-M 24B generates at roughly ~28% of its short-context speed. How this is estimated.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 10.1 GB est
Q3_K_M 11.7 GB est
Q4_K_M (default) 14.3 GB
Q5_K_M 17.1 GB est
Q6_K 19.7 GB est
Q8_0 25.1 GB
FP16 47.2 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

llama.cpp
$ llama-cli -hf lmstudio-community/sarvam-m-GGUF:Q4_K_M
LM Studio
$ lms get lmstudio-community/sarvam-m-GGUF

Which devices can run Sarvam-M 24B?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down ERNIE 4.5 21B-A3B 21B Next size up Gemma 4 26B-A4B 26.5B

Similar models

Head-to-head

FAQ

How much VRAM or RAM does Sarvam-M 24B need?

At Q4_K_M, Sarvam-M 24B needs about 16.3 GB (weights ~14.3 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~27.1 GB.

What is the Q4_K_M GGUF file size of Sarvam-M 24B?

The Q4_K_M GGUF file is about 14.3 GB to download, and the Q8_0 GGUF is about 25.1 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~16.3 GB of memory at Q4_K_M.

Can Sarvam-M 24B run on a laptop?

Sarvam-M 24B is large; you need a 24 GB+ GPU or a 32-48 GB Mac at Q4_K_M.

Can I use Sarvam-M 24B commercially?

Yes. Sarvam-M 24B is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Sarvam-M 24B's memory and quant figures.

Dense 24B fine-tuned from Mistral-Small-3.1-24B-Base. Hybrid thinking mode. Q4_K_M 14.3GB and Q8_0 25.1GB confirmed from two independent GGUF repos (lmstudio-community, Mungert) plus the official sarvamai Q8 repo. Context 32K from config.json.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.