Model family · 3 sizes
SmolLM2: which size runs locally?
SmolLM2 comes in 3 sizes, from 0.135B to 1.7B. Bigger is generally more capable but needs more memory. Here is each size with its Q4_K_M weight, the memory it needs, and the hardware that runs it.
- Sizes
- 3
- Smallest
- 0.135B
- Largest
- 1.7B
- Runs from
- 8GB
The SmolLM2 lineup
"Needs" is the sourced minimum memory for Q4_K_M with a small context. Larger context needs more.
Which SmolLM2 fits your memory
Largest that fits: SmolLM2 1.7B (1.7B), best case on Apple M1 (8GB).
Largest that fits: SmolLM2 1.7B (1.7B), best case on Nvidia GeForce RTX 4080 (16GB).
Largest that fits: SmolLM2 1.7B (1.7B), best case on Nvidia GeForce RTX 4090 (24GB).
Largest that fits: SmolLM2 1.7B (1.7B), best case on Nvidia GeForce RTX 5090 (32GB).
Best case means the most capable device at that size (usually a discrete GPU). A Mac at the same size sits roughly one rung lower; see the per-size breakdown on each memory budget page.
FAQ
Which SmolLM2 size should I run locally?
Pick the largest size your memory allows. On 8GB (best case) up to SmolLM2 1.7B; On 16GB (best case) up to SmolLM2 1.7B; On 24GB (best case) up to SmolLM2 1.7B; On 32GB (best case) up to SmolLM2 1.7B. Smaller sizes run faster and leave headroom for context.
What is the smallest SmolLM2 model?
SmolLM2 135M at 0.135B parameters, about 0.105 GB on disk at Q4_K_M and roughly 1 GB of memory to run. It is the one to use on phones and 8 GB machines.
What is the largest SmolLM2 model and what does it need?
SmolLM2 1.7B at 1.7B, about 1.06 GB at Q4_K_M and roughly 3 GB of memory. It fits a high-memory desktop GPU or Mac.
Sources
Memory figures are estimates at Q4_K_M. See methodology.