Skip to content
localmodel.run

Text model · Nemotron

NE Nemotron 3.5 Bolt E4B: RAM and VRAM requirements

Nemotron 3.5 Bolt E4B needs about 8.9 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~7.37 GB to download; KV cache and overhead add the rest), or about 10.6 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 3060 (12GB).

Nemotron family · 8B params · released Oct 2026.

Shopping for hardware? See what runs Nemotron 3.5 Bolt E4B →

Vision · accepts images License OpenMDW-1.1 · Commercial OK ↓ 9.9K/mo ♥ 60 on HuggingFace
Q4_K_M GGUF
7.37 GB
Q8_0 GGUF
9.09 GB
Memory @ Q4 (4k)
~8.9 GB
Context
64 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Nemotron 3.5 Bolt E4B runs on 35 of 49 tracked devices at Q4_K_M.

35 run well 0 tight fit 14 too small
Fit check Q4_K_M
Yes, it runs fast
needs 8.9 GB usable 10.5 GB
See the full breakdown
$ollama run nemotron-3.5-bolt:e4b-q4_K_M

Memory breakdown

Weights (Q4_K_M)7.37 GB
+
KV cache (4k)0.7 GB
+
Overhead0.8 GB
=
Total8.9 GB

How context length changes it

4k context ~8.9 GB 32k context ~13.6 GB 128k context ~29.9 GB

Longer context grows the KV cache quickly: Nemotron 3.5 Bolt E4B needs ~8.9 GB at 4k but ~29.9 GB at 128k, which can push it past a device that fits it at a short context.

Speed drops too: every token re-reads the KV cache, so at 128k context Nemotron 3.5 Bolt E4B generates at roughly ~25% of its short-context speed. How this is estimated.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 3.4 GB est
Q3_K_M 3.9 GB est
Q4_K_M (default) 7.37 GB
Q5_K_M 5.7 GB est
Q6_K 6.6 GB est
Q8_0 9.09 GB
FP16 15.63 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

Ollama
$ ollama run nemotron-3.5-bolt:e4b-q4_K_M

Which devices can run Nemotron 3.5 Bolt E4B?

NVIDIA GPUs

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Olmo 3 7B Instruct 7B Next size up DeepSeek-R1-0528-Qwen3-8B 8.19B

Similar models

Head-to-head

FAQ

How much VRAM or RAM does Nemotron 3.5 Bolt E4B need?

At Q4_K_M, Nemotron 3.5 Bolt E4B needs about 8.9 GB (weights ~7.37 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~10.6 GB.

What is the Q4_K_M GGUF file size of Nemotron 3.5 Bolt E4B?

The Q4_K_M GGUF file is about 7.37 GB to download, and the Q8_0 GGUF is about 9.09 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~8.9 GB of memory at Q4_K_M.

Can Nemotron 3.5 Bolt E4B run on a laptop?

Yes, Nemotron 3.5 Bolt E4B fits on a 16 GB machine at Q4_K_M and runs on Apple Silicon or a 12 GB+ GPU comfortably.

Can I use Nemotron 3.5 Bolt E4B commercially?

Yes. Nemotron 3.5 Bolt E4B is licensed OpenMDW-1.1, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Nemotron 3.5 Bolt E4B's memory and quant figures.

Dense Gemma 4 derivative with 8B total and 4B effective parameters, not a MoE. Measured Ollama GGUF totals include the model, combined vision/audio projector (991,551,808 bytes), and default DFlash draft (736,812,096 bytes): Q4_K_M 7,908,810,112 bytes, Q8_0 9,759,607,744 bytes, BF16 16,781,462,464 bytes, converted to GiB. The NVIDIA quant repository contains only the main model; projector and draft sizes come from the Ollama manifests. Vendor context is 65,536 tokens; the Ollama 128K label is not used. The bare Ollama slug selects a different 74B IQ2 model; use the explicit E4B tag.

Sources

Catalog updated 2026-11-01. Memory figures are estimates. See methodology.