Text model · Olmo
OL Olmo 3.1 32B Instruct: RAM and VRAM requirements
Olmo 3.1 32B Instruct needs about 20.3 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~18.14 GB to download; KV cache and overhead add the rest), or about 34.1 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 4090 (24GB).
Olmo family · 32B params · released Dec 2025.
Shopping for hardware? See what runs Olmo 3.1 32B Instruct →
Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.
Will it run on your device?
Olmo 3.1 32B Instruct runs on 13 of 40 tracked devices at Q4_K_M.
Memory breakdown
How context length changes it
Longer context grows the KV cache quickly: Olmo 3.1 32B Instruct needs ~20.3 GB at 4k but ~62.3 GB at 128k, which can push it past a device that fits it at a short context.
Speed drops too: every token re-reads the KV cache, so at 128k context Olmo 3.1 32B Instruct generates at roughly ~29% of its short-context speed. How this is estimated.
Quantization sizes
| Quantization | Size on disk |
|---|---|
| Q2_K | 13.4 GB est |
| Q3_K_M | 15.6 GB est |
| Q4_K_M (default) | 18.14 GB |
| Q5_K_M | 22.8 GB est |
| Q6_K | 26.2 GB est |
| Q8_0 | 31.9 GB |
| FP16 | 64 GB est |
Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.
Run it
ollama run olmo-3.1:32b-instruct llama-cli -hf bartowski/allenai_Olmo-3.1-32B-Instruct-GGUF:Q4_K_M lms get bartowski/allenai_Olmo-3.1-32B-Instruct-GGUF Which devices can run Olmo 3.1 32B Instruct?
Apple Silicon Macs
- Apple M1 (8GB) No
- Apple M2 (16GB) No
- Apple M4 (16GB) No
- Apple M5 (16GB) No
- Apple M3 Pro (18GB) No
- Apple M4 (24GB) No
- Apple M4 Pro (24GB) No
- Apple M5 (32GB) Tight
- Apple M4 Pro (48GB) Yes
- Apple M5 Pro (48GB) Yes
- Apple M4 Max (64GB) Yes
- Apple M4 Max (128GB) Yes
- Apple M5 Max (128GB) Yes
- Apple M3 Ultra (256GB) Yes
RAM-only laptops
iPhone & iPad
Android
NVIDIA GPUs
Same job, different size
Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.
Similar models
FAQ
How much VRAM or RAM does Olmo 3.1 32B Instruct need?
At Q4_K_M, Olmo 3.1 32B Instruct needs about 20.3 GB (weights ~18.14 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~34.1 GB.
What is the Q4_K_M GGUF file size of Olmo 3.1 32B Instruct?
The Q4_K_M GGUF file is about 18.14 GB to download, and the Q8_0 GGUF is about 31.9 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~20.3 GB of memory at Q4_K_M.
Can Olmo 3.1 32B Instruct run on a laptop?
Olmo 3.1 32B Instruct is large; you need a 24 GB+ GPU or a 32-48 GB Mac at Q4_K_M.
Can I use Olmo 3.1 32B Instruct commercially?
Yes. Olmo 3.1 32B Instruct is licensed Apache-2.0, which permits commercial use.
Understand the numbers
Short guides to the ideas behind Olmo 3.1 32B Instruct's memory and quant figures.
AI2's fully-open dense 32B, instruct-tuned. A matching -Think reasoning variant ships at the same size but isn't cataloged as a separate row here. 64K context (config max_position_embeddings). Fully open data, code, and checkpoints. Apache-2.0.
Sources
Last validated 2026-08-03. Memory figures are estimates. See methodology.