Skip to content
localmodel.run

Text model · SmolLM2

SmolLM2 360M: RAM and VRAM requirements

SmolLM2 360M needs about 1.2 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~0.271 GB to download; KV cache and overhead add the rest), or about 1.3 GB at Q8_0. The lightest hardware that runs it is Apple M1 (8GB).

SmolLM2 family · 0.362B params · released 2024-11-02 · 2.6M Ollama pulls.

Shopping for hardware? See what runs SmolLM2 360M →

License Apache-2.0 · Commercial OK ↓ 506.5K/mo ♥ 205 on HuggingFace
Q4_K_M GGUF
0.271 GB
Q8_0 GGUF
0.386 GB
Memory @ Q4 (4k)
~1.2 GB
Context
2 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

SmolLM2 360M runs on 40 of 40 tracked devices at Q4_K_M.

40 run well 0 tight fit 0 too small
Fit check Q4_K_M
Yes, it runs fast
needs 1.2 GB usable 10.5 GB
See the full breakdown
$ollama run smollm2:360m

Memory breakdown

Weights (Q4_K_M)0.271 GB
+
KV cache (4k)0.1 GB
+
Overhead0.8 GB
=
Total1.2 GB

How context length changes it

4k context ~1.2 GB 32k context ~2.3 GB 128k context ~5.7 GB

SmolLM2 360M is small enough that the context window is the thing to size for: 128k needs ~5.7 GB versus ~1.2 GB at 4k, so pick the window you actually use.

Speed drops too: every token re-reads the KV cache, so at 128k context SmolLM2 360M generates at roughly ~6% of its short-context speed. How this is estimated.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 0.2 GB est
Q3_K_M 0.2 GB est
Q4_K_M (default) 0.271 GB
Q5_K_M 0.3 GB est
Q6_K 0.3 GB est
Q8_0 0.386 GB
FP16 0.7 GB est

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

Ollama
$ ollama run smollm2:360m
llama.cpp
$ llama-cli -hf bartowski/SmolLM2-360M-Instruct-GGUF:Q4_K_M
LM Studio
$ lms get bartowski/SmolLM2-360M-Instruct-GGUF

Which devices can run SmolLM2 360M?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down LFM2 350M 354M Next size up Qwen2.5 0.5B 494M

Similar models

Head-to-head

FAQ

How much VRAM or RAM does SmolLM2 360M need?

At Q4_K_M, SmolLM2 360M needs about 1.2 GB (weights ~0.271 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~1.3 GB.

What is the Q4_K_M GGUF file size of SmolLM2 360M?

The Q4_K_M GGUF file is about 0.271 GB to download, and the Q8_0 GGUF is about 0.386 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~1.2 GB of memory at Q4_K_M.

Can SmolLM2 360M run on a laptop?

Yes, SmolLM2 360M fits on a 16 GB machine at Q4_K_M and runs on Apple Silicon or a 12 GB+ GPU comfortably.

Can I use SmolLM2 360M commercially?

Yes. SmolLM2 360M is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind SmolLM2 360M's memory and quant figures.

Mid-tier SmolLM2. 362M params, 271MB at Q4_K_M. Trained on 4T tokens. Suitable for phones and embedded Linux devices. Apache 2.0 license.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.