Skip to content
localmodel.run

Model family · 4 sizes

Nemotron: which size runs locally?

Nemotron comes in 4 sizes, from 8B to 120B. Some sizes are Mixture-of-Experts, so they run faster than their memory footprint suggests. Here is each size with its Q4_K_M weight, the memory it needs, and the hardware that runs it.

Sizes
4
Smallest
8B
Largest
120B
Runs from
12GB

The Nemotron lineup

Smallest first Q4_K_M · min memory

"Needs" is the sourced minimum memory for Q4_K_M with a small context. Larger context needs more.

Which Nemotron fits your memory

6GB

No Nemotron size fits 6GB; even Nemotron 3.5 Bolt E4B needs more.

No
8GB

No Nemotron size fits 8GB; even Nemotron 3.5 Bolt E4B needs more.

No
12GB

Largest that fits: Nemotron Nano 9B v2 (9B), best case on Nvidia GeForce RTX 5070 (12GB).

Yes
16GB

Largest that fits: Nemotron Nano 9B v2 (9B), best case on Nvidia GeForce RTX 5080 (16GB).

Yes
24GB

Largest that fits: Nemotron Nano 9B v2 (9B), best case on Nvidia GeForce RTX 4090 (24GB).

Yes
32GB

Largest that fits: Llama-3.3-Nemotron-Super-49B-v1 (49B), best case on Nvidia GeForce RTX 5090 (32GB). Comfortable up to Nemotron Nano 9B v2 (9B).

Tight
48GB

Largest that fits: Llama-3.3-Nemotron-Super-49B-v1 (49B), best case on Apple M5 Pro (48GB). Comfortable up to Nemotron Nano 9B v2 (9B).

Tight
64GB

Largest that fits: Llama-3.3-Nemotron-Super-49B-v1 (49B), best case on Apple M4 Max (64GB).

Yes
128GB

Largest that fits: Nemotron 3.5 Super VL 120B-A12B (120B), best case on Apple M5 Max (128GB). Comfortable up to Llama-3.3-Nemotron-Super-49B-v1 (49B).

Tight
256GB

Largest that fits: Nemotron 3.5 Super VL 120B-A12B (120B), best case on Apple M3 Ultra (256GB).

Yes

Best case means the most capable device at that size (usually a discrete GPU). A Mac at the same size sits roughly one rung lower; see the per-size breakdown on each memory budget page.

FAQ

Which Nemotron size should I run locally?

Pick the largest size your memory allows. On 12GB (best case) up to Nemotron Nano 9B v2; On 16GB (best case) up to Nemotron Nano 9B v2; On 24GB (best case) up to Nemotron Nano 9B v2; On 32GB (best case) up to Llama-3.3-Nemotron-Super-49B-v1; On 48GB (best case) up to Llama-3.3-Nemotron-Super-49B-v1; On 64GB (best case) up to Llama-3.3-Nemotron-Super-49B-v1; On 128GB (best case) up to Nemotron 3.5 Super VL 120B-A12B; On 256GB (best case) up to Nemotron 3.5 Super VL 120B-A12B. Smaller sizes run faster and leave headroom for context.

What is the smallest Nemotron model?

Nemotron 3.5 Bolt E4B at 8B parameters, about 7.37 GB on disk at Q4_K_M. It is the one to use on phones and 8 GB machines.

What is the largest Nemotron model and what does it need?

Nemotron 3.5 Super VL 120B-A12B at 120B (mixture of experts), about 83.96 GB at Q4_K_M. It fits a high-memory desktop GPU or Mac.

Understand the numbers

Short guides to the ideas behind Nemotron's memory and quant figures.

Sources

Memory figures are estimates at Q4_K_M. See methodology.