Skip to content
localmodel.run

Text model · Nemotron 3.5

NE Nemotron 3.5 Lightning 30B-A3B: RAM and VRAM requirements

Nemotron 3.5 Lightning 30B-A3B needs about 25.8 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~23.73 GB to download; KV cache and overhead add the rest), or about 36.7 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 5090 (32GB).

Nemotron 3.5 family · 30B params (Mixture-of-Experts: activates only 3B of 30B params per token, so generation is faster than the total size suggests) · released Aug 2026 · 139.8K Ollama pulls.

Shopping for hardware? See what runs Nemotron 3.5 Lightning 30B-A3B →

License OpenMDW-1.1 · Commercial OK ↓ 475K/mo ♥ 202 on HuggingFace
Q4_K_M GGUF
23.73 GB
Q8_0 GGUF
34.63 GB
Memory @ Q4 (4k)
~25.8 GB
Context
1000 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Nemotron 3.5 Lightning 30B-A3B runs on 9 of 43 tracked devices at Q4_K_M.

8 run well 1 tight fit 34 too small
Fit check Q4_K_M
No, not enough memory
needs 25.8 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)23.73 GB
+
KV cache (4k)1.3 GB
+
Overhead0.8 GB
=
Total25.8 GB

How context length changes it

4k context ~25.8 GB 32k context ~35 GB 128k context ~66.6 GB

Longer context grows the KV cache, which for Nemotron 3.5 Lightning 30B-A3B is sized by its 3B active params, not the full 30B. It needs ~25.8 GB at 4k and ~66.6 GB at 128k.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 12.6 GB est
Q3_K_M 14.7 GB est
Q4_K_M (default) 23.73 GB
Q5_K_M 21.4 GB est
Q6_K 24.6 GB est
Q8_0 34.63 GB
FP16 65.85 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

Ollama
$ ollama run nemotron-3.5-lightning:30b-a3b
llama.cpp
$ llama-cli -hf bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_K_M
LM Studio
$ lms get bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF

Which devices can run Nemotron 3.5 Lightning 30B-A3B?

NVIDIA GPUs

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Granite 4.1 30B 28.9B Next size up Qwen3 30B-A3B 30.5B

Similar models

FAQ

How much VRAM or RAM does Nemotron 3.5 Lightning 30B-A3B need?

At Q4_K_M, Nemotron 3.5 Lightning 30B-A3B needs about 25.8 GB (weights ~23.73 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~36.7 GB.

What is the Q4_K_M GGUF file size of Nemotron 3.5 Lightning 30B-A3B?

The Q4_K_M GGUF file is about 23.73 GB to download, and the Q8_0 GGUF is about 34.63 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~25.8 GB of memory at Q4_K_M.

Can Nemotron 3.5 Lightning 30B-A3B run on a laptop?

Nemotron 3.5 Lightning 30B-A3B is large; you need a high-memory Mac or multi-GPU setup at Q4_K_M.

Is Nemotron 3.5 Lightning 30B-A3B cheaper to run because it is a MoE model?

It is faster, not lighter. Nemotron 3.5 Lightning 30B-A3B activates only 3B of 30B params per token (so it runs quickly), but all experts must stay in memory, so it still needs memory for the full 30B.

Can I use Nemotron 3.5 Lightning 30B-A3B commercially?

Yes. Nemotron 3.5 Lightning 30B-A3B is licensed OpenMDW-1.1, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Nemotron 3.5 Lightning 30B-A3B's memory and quant figures.

NVIDIA's Nemotron 3.5 generation: 30B total / 3B active, 1M context, OpenMDW-1.1. Quants run heavier than the older Nano 30B-A3B (the bf16 is 65.85GB, so Q4_K_M lands at 25.48GB). Sizes from bartowski's repo, main files only (the separate 2.18GB MTP draft GGUF is excluded).

Sources

Last validated 2026-09-07. Memory figures are estimates. See methodology.