Skip to content
localmodel.run

Text model · OLMo

OL OLMo 2 32B Instruct: RAM and VRAM requirements

OLMo 2 32B Instruct needs about 20.3 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~18.14 GB to download; KV cache and overhead add the rest), or about 34.1 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 4090 (24GB).

OLMo family · 32B params · released Mar 2025.

Shopping for hardware? See what runs OLMo 2 32B Instruct →

Set up OLMo 2 32B Instruct on your hardware →

License Apache-2.0 · Commercial OK ↓ 9K/mo ♥ 148 on HuggingFace
Q4_K_M GGUF
18.14 GB
Q8_0 GGUF
31.9 GB
Memory @ Q4 (4k)
~20.3 GB
Context
4 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

OLMo 2 32B Instruct runs on 14 of 49 tracked devices at Q4_K_M.

13 run well 1 tight fit 35 too small
Fit check Q4_K_M
No, not enough memory
needs 20.3 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)18.14 GB
+
KV cache (4k)1.4 GB
+
Overhead0.8 GB
=
Total20.3 GB

How context length changes it

4k context ~20.3 GB 32k context ~29.8 GB 128k context ~62.3 GB

Longer context grows the KV cache quickly: OLMo 2 32B Instruct needs ~20.3 GB at 4k but ~62.3 GB at 128k, which can push it past a device that fits it at a short context.

Speed drops too: every token re-reads the KV cache, so at 128k context OLMo 2 32B Instruct generates at roughly ~29% of its short-context speed. How this is estimated.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 13.4 GB est
Q3_K_M 15.6 GB est
Q4_K_M (default) 18.14 GB
Q5_K_M 22.8 GB est
Q6_K 26.2 GB est
Q8_0 31.9 GB
FP16 60.05 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

llama.cpp
$ llama-cli -hf unsloth/OLMo-2-0325-32B-Instruct-GGUF:Q4_K_M
LM Studio
$ lms get unsloth/OLMo-2-0325-32B-Instruct-GGUF

Which devices can run OLMo 2 32B Instruct?

NVIDIA GPUs

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Qwen3-VL 30B-A3B 31.1B Next size up Gemma 4 31B 32.7B

Similar models

FAQ

How much VRAM or RAM does OLMo 2 32B Instruct need?

At Q4_K_M, OLMo 2 32B Instruct needs about 20.3 GB (weights ~18.14 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~34.1 GB.

What is the Q4_K_M GGUF file size of OLMo 2 32B Instruct?

The Q4_K_M GGUF file is about 18.14 GB to download, and the Q8_0 GGUF is about 31.9 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~20.3 GB of memory at Q4_K_M.

Can OLMo 2 32B Instruct run on a laptop?

OLMo 2 32B Instruct is large; you need a 24 GB+ GPU or a 32-48 GB Mac at Q4_K_M.

Can I use OLMo 2 32B Instruct commercially?

Yes. OLMo 2 32B Instruct is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind OLMo 2 32B Instruct's memory and quant figures.

AI2's fully open 32B (weights, training data and code all public). Note the small 4K context. Q4_K_M, Q8_0 and BF16 sizes are measured GGUF bytes from unsloth/OLMo-2-0325-32B-Instruct-GGUF, converted to GiB (revision c10a156f8a43b8f175c306dc5e341281eb2f7cf0).

Sources

Catalog updated 2026-11-01. Memory figures are estimates. See methodology.