Skip to content
localmodel.run

Text model · Sarvam

S Sarvam-105B: RAM and VRAM requirements

Sarvam-105B needs about 67.5 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~64.2 GB to download; KV cache and overhead add the rest), or about 114.9 GB at Q8_0. The lightest hardware that runs it is Apple M4 Max (128GB).

Sarvam family · 105B params (Mixture-of-Experts: activates only 10.3B of 105B params per token, so generation is faster than the total size suggests) · released Mar 2026.

Shopping for hardware? See what runs Sarvam-105B →

License Apache-2.0 · Commercial OK ↓ 57K/mo ♥ 283 on HuggingFace
Q4_K_M GGUF
64.2 GB
Q8_0 GGUF
-
Memory @ Q4 (4k)
~67.5 GB
Context
128 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Sarvam-105B runs on 4 of 40 tracked devices at Q4_K_M.

4 run well 0 tight fit 36 too small
Fit check Q4_K_M
No, not enough memory
needs 67.5 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)64.2 GB
+
KV cache (4k)2.5 GB
+
Overhead0.8 GB
=
Total67.5 GB

How context length changes it

4k context ~67.5 GB 32k context ~84.7 GB 128k context ~143.7 GB

Longer context grows the KV cache, which for Sarvam-105B is sized by its 10.3B active params, not the full 105B. It needs ~67.5 GB at 4k and ~143.7 GB at 128k.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 44 GB est
Q3_K_M 51.3 GB est
Q4_K_M (default) 64.2 GB
Q5_K_M 74.8 GB est
Q6_K 86.1 GB est
Q8_0 111.6 GB est
FP16 210 GB est

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

llama.cpp
$ llama-cli -hf sarvamai/sarvam-105b-gguf:Q4_K_M
LM Studio
$ lms get sarvamai/sarvam-105b-gguf

Which devices can run Sarvam-105B?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Hunyuan-A13B-Instruct 80B Next size up GLM-4.5-Air 106B

Similar models

Head-to-head

FAQ

How much VRAM or RAM does Sarvam-105B need?

At Q4_K_M, Sarvam-105B needs about 67.5 GB (weights ~64.2 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~114.9 GB.

What is the Q4_K_M GGUF file size of Sarvam-105B?

The Q4_K_M GGUF file is about 64.2 GB to download, and the Q8_0 GGUF is about 111.6 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~67.5 GB of memory at Q4_K_M.

Can Sarvam-105B run on a laptop?

Sarvam-105B is large; you need a high-memory Mac or multi-GPU setup at Q4_K_M.

Is Sarvam-105B cheaper to run because it is a MoE model?

It is faster, not lighter. Sarvam-105B activates only 10.3B of 105B params per token (so it runs quickly), but all experts must stay in memory, so it still needs memory for the full 105B.

Can I use Sarvam-105B commercially?

Yes. Sarvam-105B is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Sarvam-105B's memory and quant figures.

MoE with 128 experts, top-8 routing plus one shared expert, 10.3B active params. Released 2026-03 under Apache 2.0. Q4_K_M 64.2GB confirmed by summing the 9 shards in the official GGUF repo. Server-class; no Q8_0 GGUF exists.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.