Device profile · Windows
Best local LLMs for Nvidia GeForce RTX 4090 (24GB)
Nvidia GeForce RTX 4090 (24GB) has ~23 GB estimated usable for weights, cache and runtime; 120 of 161 tracked models fit at the estimated 4K baseline. Best tool: LM Studio.
107 fit with room 13 tight fit 41 too heavy
- Usable memory
- ~23 GB
- Estimated fits
- 120
- Too large
- 41
- Suggested fit
- 27B
Suggested fit Q4_K_M
Gemma 3 27B Yes
Runs at Q4_K_M using ~18.6 GB of ~23 GB usable.
The suggested model uses recorded Arena scores where available, then estimated fit and speed. Score coverage is partial; it does not establish the best model for every task. Fit is an estimate, not a benchmark on this device.
Set up Gemma 3 27B on this device →
Estimated memory fits on Nvidia GeForce RTX 4090 (24GB)
Compatible models 120 total
- SE TightSeed-OSS 36B Instruct36B · ~22.5 GB at Q4_K_M
- CR TightCommand R 35B35B · ~22.3 GB at Q4_K_M
- OR TightOrnith 1.5 35B-A3B35B MoE · ~22.4 GB at Q4_K_M
- Yi TightYi 1.5 34B34B · ~21.4 GB at Q4_K_M
- FN TightFalcon-H1-34B-Instruct34B · ~21.1 GB at Q4_K_M
- LA TightLaguna XS 2.133.4B MoE · ~22.2 GB at Q4_K_M
- YesGemma 4 31B32.7B · ~20.5 GB at Q4_K_M
- TightQwen2.5 32B32B · ~22.1 GB at Q4_K_M
- TightQwen3 32B32B · ~22 GB at Q4_K_M
- TightDeepSeek-R1-Distill-Qwen 32B32B · ~22.1 GB at Q4_K_M
- TightQwen2.5 Coder 32B32B · ~20.7 GB at Q4_K_M
- YesGranite 4.0 H Small32B MoE · ~20.4 GB at Q4_K_M
- GL YesGLM-4-32B-041432B · ~20.5 GB at Q4_K_M
- EX YesEXAONE 4.0 32B32B · ~20.2 GB at Q4_K_M
- OL YesOLMo 2 32B Instruct32B · ~20.3 GB at Q4_K_M
- OL YesOlmo 3.1 32B Instruct32B · ~20.3 GB at Q4_K_M
- YesQwen3-VL 30B-A3B31.1B MoE · ~20.4 GB at Q4_K_M
- TightQwen3 30B-A3B30.5B MoE · ~20.7 GB at Q4_K_M
- YesQwen3-Coder 30B-A3B30.5B MoE · ~19.4 GB at Q4_K_M
- NO TightNorth Mini Code 1.030.5B MoE · ~21.1 GB at Q4_K_M
- S TightSarvam-30B30B MoE · ~21.7 GB at Q4_K_M
- GL YesGLM-4.7-Flash30B MoE · ~19.2 GB at Q4_K_M
- MU YesMuse Glimmer 30B30B · ~18.2 GB at Q4_K_M
- YesGranite 4.2 30B30B · ~18.6 GB at Q4_K_M
- YesGranite 4.1 30B28.9B · ~19.1 GB at Q4_K_M
- YesQwen3.6 27B27.8B · ~18.9 GB at Q4_K_M
- YesGemma 2 27B27B · ~18.7 GB at Q4_K_M
- YesGemma 3 27B27B · ~18.6 GB at Q4_K_M
- YesQwen3.8 27B27B · ~17.3 GB at Q4_K_M
- YesGemma 4 26B-A4B26.5B MoE · ~19 GB at Q4_K_M
- YesMistral Small 3 24B24B · ~16.3 GB at Q4_K_M
- S YesSarvam-M 24B24B · ~16.3 GB at Q4_K_M
- YesMistral Small 3.1 24B24B · ~15.4 GB at Q4_K_M
- YesMagistral Small24B · ~16 GB at Q4_K_M
- YesDevstral Small24B · ~15.4 GB at Q4_K_M
- LF YesLFM2 24B-A2B24B MoE · ~15.4 GB at Q4_K_M
- YesDevstral Small 2 24B24B · ~15.4 GB at Q4_K_M
- Yesgpt-oss 20B21B MoE · ~13.2 GB at Q4_K_M
- ER YesERNIE 4.5 21B-A3B21B MoE · ~14.3 GB at Q4_K_M
- YesDeepSeek-V2-Lite16B MoE · ~12.2 GB at Q4_K_M
- YesPhi-4 14B14B · ~10.8 GB at Q4_K_M
- YesQwen2.5 14B14B · ~10.7 GB at Q4_K_M
- YesQwen3 14B14B · ~10.7 GB at Q4_K_M
- YesDeepSeek-R1-Distill-Qwen 14B14B · ~10.7 GB at Q4_K_M
- YesQwen2.5 Coder 14B14B · ~10.1 GB at Q4_K_M
- YesPhi-4-reasoning14B · ~10.1 GB at Q4_K_M
- YesMinistral 3 14B14B · ~9.4 GB at Q4_K_M
- YesMistral Nemo 12B12.2B · ~8.6 GB at Q4_K_M
- YesGemma 3 12B12B · ~8.9 GB at Q4_K_M
- YesGemma 4 12B12B · ~8.7 GB at Q4_K_M
- YesLlama 3.2 Vision 11B10.7B · ~9 GB at Q4_K_M
- FN YesFalcon3 10B10B · ~7.5 GB at Q4_K_M
- YesGemma 2 9B9B · ~7.3 GB at Q4_K_M
- GL YesGLM-4 9B9B · ~7.3 GB at Q4_K_M
- GL YesGLM-4-9B-04149B · ~7.2 GB at Q4_K_M
- NE YesNemotron Nano 9B v29B · ~7.6 GB at Q4_K_M
- OR YesOrnith 1.0 9B9B · ~7.1 GB at Q4_K_M
- OR YesOrnith 1.5 9B9B · ~6.9 GB at Q4_K_M
- YesGranite 4.1 8B8.8B · ~6.8 GB at Q4_K_M
- YesQwen3-VL 8B8.77B · ~7.3 GB at Q4_K_M
- LF YesLFM2.5 8B-A1B8.3B MoE · ~6.3 GB at Q4_K_M
- YesQwen2.5-VL 7B8.29B · ~7.1 GB at Q4_K_M
- YesDeepSeek-R1-0528-Qwen3-8B8.19B · ~6.2 GB at Q4_K_M
- YesLlama 3.1 8B8B · ~6.4 GB at Q4_K_M
- YesQwen3 8B8B · ~6.5 GB at Q4_K_M
- YesDeepSeek-R1-Distill-Llama 8B8B · ~6.4 GB at Q4_K_M
- YesGemma 3n E4B8B · ~5.7 GB at Q4_K_M
- YesGemma 4 E4B8B · ~6.5 GB at Q4_K_M
- YesMinistral 3 8B8B · ~6.3 GB at Q4_K_M
- RN YesRNJ-1 8B8B · ~6.3 GB at Q4_K_M
- YesGranite 4.2 8B8B · ~6.5 GB at Q4_K_M
- NE YesNemotron 3.5 Bolt E4B8B · ~8.9 GB at Q4_K_M
- YesMistral 7B7B · ~5.8 GB at Q4_K_M
- YesQwen2.5 7B7B · ~6.1 GB at Q4_K_M
- YesDeepSeek-R1-Distill-Qwen 7B7B · ~6.1 GB at Q4_K_M
- YesQwen2.5 Coder 7B7B · ~5.8 GB at Q4_K_M
- OL YesOlmo 3 7B Instruct7B · ~5.6 GB at Q4_K_M
- YesGemma 4 E2B5.1B · ~4.4 GB at Q4_K_M
- YesQwen3-VL 4B4.44B · ~4.4 GB at Q4_K_M
- YesGemma 3 4B4B · ~3.8 GB at Q4_K_M
- YesQwen3 4B4B · ~3.8 GB at Q4_K_M
- NE YesNemotron 3 Nano 4B4B · ~3.9 GB at Q4_K_M
- YesPhi-3.5-mini 3.8B3.82B · ~3.7 GB at Q4_K_M
- YesPhi-4-mini 3.8B3.8B · ~3.8 GB at Q4_K_M
- YesPhi-4-mini-reasoning3.8B · ~3.6 GB at Q4_K_M
- YesQwen2.5-VL 3B3.75B · ~4.4 GB at Q4_K_M
- YesGranite 4.1 3B3.4B · ~3.3 GB at Q4_K_M
- YesQwen2.5 3B3.09B · ~3.3 GB at Q4_K_M
- YesQwen2.5 Coder 3B3.09B · ~3 GB at Q4_K_M
- YesLlama 3.2 3B3B · ~3.2 GB at Q4_K_M
- YesSmolLM3 3B3B · ~3 GB at Q4_K_M
- OP YesApple OpenELM 3B3B · ~3 GB at Q4_K_M
- YesMinistral 3 3B3B · ~3.2 GB at Q4_K_M
- YesGranite 4.2 3B3B · ~3.3 GB at Q4_K_M
- YesGemma 2 2B2.61B · ~2.9 GB at Q4_K_M
- YesGranite 3.1 2B2.53B · ~2.8 GB at Q4_K_M
- LF YesLFM3 Omni 2B2.17B · ~3.1 GB at Q4_K_M
- S YesSarvam-1 2B2B · ~2.7 GB at Q4_K_M
- LF YesLFM3-VL-2B1.92B · ~2.5 GB at Q4_K_M
- LF YesLFM3-1.7B1.73B · ~2.1 GB at Q4_K_M
- YesSmolLM2 1.7B1.7B · ~2.2 GB at Q4_K_M
- YesQwen3 1.7B1.7B · ~2.4 GB at Q4_K_M
- YesQwen2.5 1.5B1.54B · ~2.2 GB at Q4_K_M
- YesQwen2.5 Coder 1.5B1.54B · ~2 GB at Q4_K_M
- MI YesMiniCPM-V 4.61.3B · ~1.6 GB at Q4_K_M
- LF YesLFM2 1.2B1.17B · ~2.5 GB at Q4_K_M
- LF YesLFM2.5 1.2B1.17B · ~1.8 GB at Q4_K_M
- LF YesLFM2.5 1.2B Thinking1.17B · ~1.8 GB at Q4_K_M
- TL YesTinyLlama 1.1B1.1B · ~1.8 GB at Q4_K_M
- OP YesApple OpenELM 1.1B1.1B · ~1.7 GB at Q4_K_M
- YesLlama 3.2 1B1B · ~1.8 GB at Q4_K_M
- YesGemma 3 1B1B · ~1.8 GB at Q4_K_M
- LF YesLFM2 700M0.742B · ~1.9 GB at Q4_K_M
- YesQwen3 0.6B0.6B · ~1.5 GB at Q4_K_M
- YesQwen2.5 0.5B0.494B · ~1.5 GB at Q4_K_M
- YesQwen2.5 Coder 0.5B0.494B · ~1.4 GB at Q4_K_M
- YesSmolLM2 360M0.362B · ~1.2 GB at Q4_K_M
- LF YesLFM2 350M0.354B · ~1.4 GB at Q4_K_M
- YesGemma 3 270M0.27B · ~1.1 GB at Q4_K_M
- YesSmolLM2 135M0.135B · ~1 GB at Q4_K_M
Too large for this device
Kimi K3DeepSeek-V4-ProKimi K2 InstructKimi K2.6Kimi K2.7 CodeGLM-5.2DeepSeek R1DeepSeek V3DeepSeek-R1-0528Nemotron 3 Ultra 550B-A55BQwen3-Coder 480B-A35B InstructMiniMax-M1-80kMiniMax M3Llama 4 MaverickGLM-4.6GLM-5.3-FlashDeepSeek-V4-FlashQwen3 235B A22Bdots.llm1Devstral 2 123BNemotron 3 Super 120B-A12BNemotron 3.5 Super VL 120B-A12BLaguna S 2.1gpt-oss 120BCommand ALlama 4 ScoutGLM-4.5-AirSarvam-105BHunyuan-A13B-InstructQwen3-Coder-Next 80B-A3BQwen2.5 72BLlama 3.3 70BLlama-3.3-Nemotron-Super-49B-v1Mixtral 8x7BFlint 40B-A2.5BQwen3.6 35B-A3BOrnith 1.0 35BQwen-AgentWorld 35B-A3BNemotron 3 Nano 30B-A3BNemotron Cascade 2 30B-A3BNemotron 3.5 Lightning 30B-A3B
Best way to run models on Windows
Runtime guide Windows
Beginner: LM Studio, Best GUI on Windows, auto-detects CUDA/Vulkan backends.
Power user: Ollama (CUDA), Scriptable server; CUDA path is fastest on NVIDIA.
AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.
Full Windows tool guide →FAQ
What is the best local LLM for Nvidia GeForce RTX 4090 (24GB)?
Gemma 3 27B is a suggested fit, using ~18.6 GB at Q4_K_M of the ~23 GB usable on Nvidia GeForce RTX 4090 (24GB).
How much of Nvidia GeForce RTX 4090 (24GB)'s memory can I use for a model?
About 23 GB. On a discrete GPU, leave ~1 GB of VRAM for the driver and display.
Which tool should I use on Windows?
LM Studio (Best GUI on Windows, auto-detects CUDA/Vulkan backends.) or Ollama (CUDA) for speed. AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.
Compare nearby devices
Hardware with a similar usable pool, and how many of the 161 tracked models each runs next to Nvidia GeForce RTX 4090 (24GB)'s 120.
32GB RAM Laptop (CPU/iGPU only)
~28 GB usable · runs 127 of 161
Nvidia GeForce RTX 3090 (24GB)
~23 GB usable · runs 120 of 161
AMD Radeon RX 7900 XTX (24GB)
~23 GB usable · runs 120 of 161
Apple M5 (32GB)
~21 GB usable · runs 109 of 161
Apple M4 (24GB)
~16 GB usable · runs 88 of 161
Apple M4 Pro (24GB)
~16 GB usable · runs 88 of 161
Sources
-
aider.chat · 1 source
-
en.wikipedia.org · 1 source
-
gorilla.cs.berkeley.edu · 1 source
-
huggingface.co · 1 source
-
lmarena.ai · 1 source
-
lmstudio.ai · 1 source
-
nvidia.com · 2 sources
-
ollama.com · 3 sources
-
wccftech.com · 1 source
Memory figures are estimates. See methodology.