Skip to content
localmodel.run

Text model · ERNIE

ER ERNIE 4.5 21B-A3B: RAM and VRAM requirements

ERNIE 4.5 21B-A3B needs about 14.3 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~12.42 GB to download; KV cache and overhead add the rest), or about 23.5 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 4060 Ti (16GB).

ERNIE family · 21B params (Mixture-of-Experts: activates only 3B of 21B params per token, so generation is faster than the total size suggests) · released Jun 2025.

Shopping for hardware? See what runs ERNIE 4.5 21B-A3B →

License Apache-2.0 · Commercial OK ↓ 44.5K/mo ♥ 175 on HuggingFace
Q4_K_M GGUF
12.42 GB
Q8_0 GGUF
21.61 GB
Memory @ Q4 (4k)
~14.3 GB
Context
131 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

ERNIE 4.5 21B-A3B runs on 17 of 40 tracked devices at Q4_K_M.

15 run well 2 tight fit 23 too small
Fit check Q4_K_M
No, not enough memory
needs 14.3 GB usable 10.5 GB
See the full breakdown

Memory breakdown

Weights (Q4_K_M)12.42 GB
+
KV cache (4k)1.1 GB
+
Overhead0.8 GB
=
Total14.3 GB

How context length changes it

4k context ~14.3 GB 32k context ~22 GB 128k context ~48.4 GB

Longer context grows the KV cache, which for ERNIE 4.5 21B-A3B is sized by its 3B active params, not the full 21B. It needs ~14.3 GB at 4k and ~48.4 GB at 128k.

Quantization sizes

GGUF quantson disk
Quantization Size on disk
Q2_K 8.8 GB est
Q3_K_M 10.3 GB est
Q4_K_M (default) 12.42 GB
Q5_K_M 15 GB est
Q6_K 17.2 GB est
Q8_0 21.61 GB
FP16 43.7 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

llama.cpp
$ llama-cli -hf unsloth/ERNIE-4.5-21B-A3B-PT-GGUF:Q4_K_M
LM Studio
$ lms get unsloth/ERNIE-4.5-21B-A3B-PT-GGUF

Which devices can run ERNIE 4.5 21B-A3B?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down DeepSeek-V2-Lite 16B Next size up Mistral Small 3 24B 24B

Similar models

FAQ

How much VRAM or RAM does ERNIE 4.5 21B-A3B need?

At Q4_K_M, ERNIE 4.5 21B-A3B needs about 14.3 GB (weights ~12.42 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~23.5 GB.

What is the Q4_K_M GGUF file size of ERNIE 4.5 21B-A3B?

The Q4_K_M GGUF file is about 12.42 GB to download, and the Q8_0 GGUF is about 21.61 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~14.3 GB of memory at Q4_K_M.

Can ERNIE 4.5 21B-A3B run on a laptop?

ERNIE 4.5 21B-A3B is large; you need a 24 GB+ GPU or a 32-48 GB Mac at Q4_K_M.

Is ERNIE 4.5 21B-A3B cheaper to run because it is a MoE model?

It is faster, not lighter. ERNIE 4.5 21B-A3B activates only 3B of 21B params per token (so it runs quickly), but all experts must stay in memory, so it still needs memory for the full 21B.

Can I use ERNIE 4.5 21B-A3B commercially?

Yes. ERNIE 4.5 21B-A3B is licensed Apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind ERNIE 4.5 21B-A3B's memory and quant figures.

Baidu's open ERNIE 4.5 MoE: 21B total / 3B active, runs on consumer hardware. Apache-2.0.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.