Device profile · Linux

Best local LLMs for Nvidia DGX Spark (128GB)

Nvidia DGX Spark (128GB) has ~112 GB estimated usable for weights, cache and runtime; 143 of 161 tracked models fit at the estimated 4K baseline. Best tool: Ollama.

143 fit with room 0 tight fit 18 too heavy

Usable memory: ~112 GB
Estimated fits: 143
Too large: 18
Suggested fit: 30.5B

Suggested fit Q4_K_M

Qwen3 30B-A3B Yes

Runs at Q4_K_M using ~20.7 GB of ~112 GB usable. You have room for FP16 for higher quality.

The suggested model uses recorded Arena scores where available, then estimated fit and speed. Score coverage is partial; it does not establish the best model for every task. Fit is an estimate, not a benchmark on this device.

Set up Qwen3 30B-A3B on this device →

Estimated memory fits on Nvidia DGX Spark (128GB)

Compatible models 143 total

Too large for this device

Kimi K3 DeepSeek-V4-Pro Kimi K2 Instruct Kimi K2.6 Kimi K2.7 Code GLM-5.2 DeepSeek R1 DeepSeek V3 DeepSeek-R1-0528 Nemotron 3 Ultra 550B-A55B Qwen3-Coder 480B-A35B Instruct MiniMax-M1-80k MiniMax M3 Llama 4 Maverick GLM-4.6 GLM-5.3-Flash DeepSeek-V4-Flash Qwen3 235B A22B

Best way to run models on Linux

Runtime guide Linux

Beginner: Ollama, One-line install, simple single-user UX.

Power user: vLLM, Highest throughput for serving many requests; PagedAttention + continuous batching.

vLLM shines for multi-user serving/throughput. For a single local chat, Ollama or llama.cpp is simpler and lighter.

Full Linux tool guide →

FAQ

What is the best local LLM for Nvidia DGX Spark (128GB)?

Qwen3 30B-A3B is a suggested fit, using ~20.7 GB at Q4_K_M of the ~112 GB usable on Nvidia DGX Spark (128GB).

How much of Nvidia DGX Spark (128GB)'s memory can I use for a model?

About 112 GB. CPU and GPU share one memory pool; the rest stays with the operating system.

Which tool should I use on Linux?

Ollama (One-line install, simple single-user UX.) or vLLM for speed. vLLM shines for multi-user serving/throughput. For a single local chat, Ollama or llama.cpp is simpler and lighter.