Device profile · Windows

Best local LLMs for Nvidia GeForce RTX 4080 (16GB)

Nvidia GeForce RTX 4080 (16GB) has ~15 GB usable for model weights and runs 69 of 135 popular models. Best tool: LM Studio.

68 run well 1 tight fit 66 too heavy

Usable memory: ~15 GB
Models run: 69
Too large: 66
Top pick: 12B

Top pick Q4_K_M

Gemma 3 12B Yes

Runs at Q4_K_M using ~8.9 GB of ~15 GB usable. You have room for Q8_0 for higher quality.

Runs on Nvidia GeForce RTX 4080 (16GB)

Compatible models 69 total

Too large for this device

DeepSeek-V4-Pro Kimi K2 Instruct Kimi K2.6 Kimi K2.7 Code GLM-5.2 DeepSeek R1 DeepSeek V3 DeepSeek-R1-0528 Nemotron 3 Ultra 550B-A55B Qwen3-Coder 480B-A35B Instruct MiniMax-M1-80k MiniMax M3 Llama 4 Maverick GLM-4.6 DeepSeek-V4-Flash Qwen3 235B A22B dots.llm1 Nemotron 3 Super 120B-A12B Laguna S 2.1 gpt-oss 120B Command A Llama 4 Scout GLM-4.5-Air Sarvam-105B Hunyuan-A13B-Instruct Qwen2.5 72B Llama 3.3 70B Llama-3.3-Nemotron-Super-49B-v1 Mixtral 8x7B Seed-OSS 36B Instruct Qwen3.6 35B-A3B Command R 35B Ornith 1.0 35B Qwen-AgentWorld 35B-A3B Yi 1.5 34B Falcon-H1-34B-Instruct Laguna XS 2.1 Gemma 4 31B Qwen2.5 32B Qwen3 32B DeepSeek-R1-Distill-Qwen 32B Qwen2.5 Coder 32B Granite 4.0 H Small GLM-4-32B-0414 EXAONE 4.0 32B OLMo 2 32B Instruct Granite 4.0 H Small Olmo 3.1 32B Instruct Qwen3 30B-A3B Qwen3-Coder 30B-A3B North Mini Code 1.0 Sarvam-30B Nemotron 3 Nano 30B-A3B Nemotron Cascade 2 30B-A3B GLM-4.7-Flash Granite 4.1 30B Qwen3.6 27B Gemma 2 27B Gemma 3 27B Gemma 4 26B-A4B Mistral Small 3 24B Sarvam-M 24B Mistral Small 3.1 24B Magistral Small Devstral Small LFM2 24B-A2B

Best way to run models on Windows

Runtime guide Windows

Beginner: LM Studio, Best GUI on Windows, auto-detects CUDA/Vulkan backends.

Power user: Ollama (CUDA), Scriptable server; CUDA path is fastest on NVIDIA.

AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.

Full Windows tool guide →

FAQ

What is the best local LLM for Nvidia GeForce RTX 4080 (16GB)?

Gemma 3 12B is the strongest model that runs comfortably, using ~8.9 GB at Q4_K_M of the ~15 GB usable on Nvidia GeForce RTX 4080 (16GB).

How much of Nvidia GeForce RTX 4080 (16GB)'s memory can I use for a model?

About 15 GB. On a discrete GPU, leave ~1 GB of VRAM for the driver and display.

Which tool should I use on Windows?

LM Studio (Best GUI on Windows, auto-detects CUDA/Vulkan backends.) or Ollama (CUDA) for speed. AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.

Compare nearby devices

Hardware with a similar usable pool, and how many of the 135 tracked models each runs next to Nvidia GeForce RTX 4080 (16GB)'s 69.

Apple M4 (24GB)

~16 GB usable · runs 73 of 135

Apple M4 Pro (24GB)

~16 GB usable · runs 73 of 135

Nvidia GeForce RTX 4060 Ti (16GB)

~15 GB usable · runs 69 of 135

Apple M3 Pro (18GB)

~12 GB usable · runs 66 of 135

-3

16GB RAM Laptop (CPU/iGPU only)

~12 GB usable · runs 66 of 135

-3

Nvidia GeForce RTX 3060 (12GB)

~11 GB usable · runs 66 of 135

-3

Sources

Memory figures are estimates. See methodology.