Hardware guide · OLMo
OL What hardware do you need to run OLMo 2 32B Instruct?
OLMo 2 32B Instruct needs about 21.7 GB to run at Q4_K_M, so the lightest hardware that runs it is Nvidia GeForce RTX 4090 (24GB). 12 of 40 devices tested can run it.
- Needs (Q4_K_M)
- ~21.7 GB
- Devices that run it
- 12
- Too small
- 28
- Lightest that runs it
- ~23 GB
Recommended Q4_K_M
Runs at Q4_K_M using ~21.7 GB of ~31 GB usable.
What kind of hardware runs OLMo 2 32B Instruct
Of the 12 devices that run it, here is the split by hardware class.
5 discrete GPUs 6 Macs 1 laptop
Hardware that runs OLMo 2 32B Instruct
Ranked by usable memory, lightest first. Prices are approximate street prices for the device itself (a GPU is the card alone; a Mac is the whole machine). Tok/s is a bandwidth estimate; see methodology.
Compatible devices 12 total
- TightNvidia GeForce RTX 4090 (24GB) from $1,599~23 GB usable · uses ~21.7 GB · ~34 tok/s est.
- TightNvidia GeForce RTX 3090 (24GB) from $1,499~23 GB usable · uses ~21.7 GB · ~31 tok/s est.
- TightAMD Radeon RX 7900 XTX (24GB) from $999~23 GB usable · uses ~21.7 GB · ~32 tok/s est.
- Yes32GB RAM Laptop (CPU/iGPU only) from $1,100~28 GB usable · uses ~21.7 GB · ~2 tok/s est.
- YesNvidia GeForce RTX 5090 (32GB) from $1,999~31 GB usable · uses ~21.7 GB · ~60 tok/s est.
- YesApple M4 Pro (48GB) from $2,399~32 GB usable · uses ~21.7 GB · ~11 tok/s est.
- YesApple M5 Pro (48GB) from $2,199~32 GB usable · uses ~21.7 GB · ~13 tok/s est.
- YesApple M4 Max (64GB) from $3,499~48 GB usable · uses ~21.7 GB · ~22 tok/s est.
- YesApple M4 Max (128GB) from $3,499~96 GB usable · uses ~21.7 GB · ~22 tok/s est.
- YesAMD Ryzen AI Halo (128GB) from $3,999~96 GB usable · uses ~21.7 GB · ~9 tok/s est.
- YesApple M5 Max (128GB) from $3,599~96 GB usable · uses ~21.7 GB · ~25 tok/s est.
- YesApple M3 Ultra (256GB) from $3,999~192 GB usable · uses ~21.7 GB · ~34 tok/s est.
Too small for OLMo 2 32B Instruct
Apple M5 (32GB) Apple M4 (24GB) Apple M4 Pro (24GB) Nvidia GeForce RTX 4060 Ti (16GB) Nvidia GeForce RTX 4080 (16GB) Apple M3 Pro (18GB) 16GB RAM Laptop (CPU/iGPU only) iPad Pro M4 (16GB, 1TB/2TB config) Samsung Galaxy S25 Ultra (16GB, 1TB config only) Samsung Galaxy S26 Ultra (16GB, 1TB config) Nvidia GeForce RTX 3060 (12GB) Nvidia GeForce RTX 4070 (12GB) Apple M2 (16GB) Apple M4 (16GB) Google Pixel 9 Pro Apple M5 (16GB) Google Pixel 10 Pro Samsung Galaxy S24 Ultra Generic Android Phone (12GB RAM) iPhone 17 Pro iPhone Air Apple M1 (8GB) 8GB RAM Laptop (CPU/iGPU only) iPhone 15 Pro iPhone 16 iPhone 16 Pro Generic Android Phone (8GB RAM) iPhone 17
See the full OLMo 2 32B Instruct memory breakdown, or compare it against other models.
Sources
Memory at Q4_K_M (21.7 GB = 19.5 GB weights + KV cache + overhead), validated 2026-08-03. See methodology.