Text model · Seed-OSS

SE Seed-OSS 36B Instruct: RAM and VRAM requirements

Seed-OSS 36B Instruct needs about 22.5 GB to run at Q4_K_M (the Q4_K_M GGUF file is ~20.27 GB to download; KV cache and overhead add the rest), or about 38 GB at Q8_0. The lightest hardware that runs it is Nvidia GeForce RTX 4090 (24GB).

Seed-OSS family · 36B params · released Aug 2025.

Shopping for hardware? See what runs Seed-OSS 36B Instruct →

License apache-2.0 · Commercial OK ↓ 41.2K/mo ♥ 505 on HuggingFace

Q4_K_M GGUF

20.27 GB

Q8_0 GGUF

35.78 GB

Memory @ Q4 (4k)

~22.5 GB

Context

512 k

Q4_K_M and Q8_0 are GGUF download (file) sizes. Memory @ Q4 includes the KV cache and overhead, so it runs larger than the file on disk.

Will it run on your device?

Seed-OSS 36B Instruct runs on 12 of 40 tracked devices at Q4_K_M.

9 run well 3 tight fit 28 too small

Fit check Q4_K_M

Your device

Apple M4 (16GB)

No, not enough memory

needs 22.5 GB usable 10.5 GB

See the full breakdown

Memory breakdown

Weights (Q4_K_M)20.27 GB

KV cache (4k)1.4 GB

Overhead0.8 GB

Total22.5 GB

How context length changes it

4k context ~22.5 GB 32k context ~32.6 GB 128k context ~67.2 GB

Longer context grows the KV cache quickly: Seed-OSS 36B Instruct needs ~22.5 GB at 4k but ~67.2 GB at 128k, which can push it past a device that fits it at a short context.

Speed drops too: every token re-reads the KV cache, so at 128k context Seed-OSS 36B Instruct generates at roughly ~31% of its short-context speed. How this is estimated.

Quantization sizes

GGUF quantson disk

Quantization	Size on disk
Q2_K	15.1 GB est
Q3_K_M	17.6 GB est
Q4_K_M (default)	20.27 GB
Q5_K_M	25.7 GB est
Q6_K	29.5 GB est
Q8_0	35.78 GB
FP16	72.31 GB

Lower quant = smaller and faster, slightly lower quality. Q4_K_M is the common default.

Run it

llama.cpp

$ llama-cli -hf bartowski/ByteDance-Seed_Seed-OSS-36B-Instruct-GGUF:Q4_K_M

LM Studio

$ lms get bartowski/ByteDance-Seed_Seed-OSS-36B-Instruct-GGUF

Which devices can run Seed-OSS 36B Instruct?

Apple Silicon Macs

RAM-only laptops

iPhone & iPad

Android

NVIDIA GPUs

AMD GPUs

Same job, different size

Pick by what fits your memory: step down to free up VRAM, or step up if you have headroom.

Next size down Ornith 1.0 35B 35B Next size up Mixtral 8x7B 46.7B

Similar models

FAQ

How much VRAM or RAM does Seed-OSS 36B Instruct need?

At Q4_K_M, Seed-OSS 36B Instruct needs about 22.5 GB (weights ~20.27 GB + KV cache + overhead) at a 4k context. At Q8_0 budget ~38 GB.

What is the Q4_K_M GGUF file size of Seed-OSS 36B Instruct?

The Q4_K_M GGUF file is about 20.27 GB to download, and the Q8_0 GGUF is about 35.78 GB. That is the weights file on disk; to run it you also need room for the KV cache and overhead, so budget ~22.5 GB of memory at Q4_K_M.

Can Seed-OSS 36B Instruct run on a laptop?

Seed-OSS 36B Instruct is large; you need a 24 GB+ GPU or a 32-48 GB Mac at Q4_K_M.

Can I use Seed-OSS 36B Instruct commercially?

Yes. Seed-OSS 36B Instruct is licensed apache-2.0, which permits commercial use.

Understand the numbers

Short guides to the ideas behind Seed-OSS 36B Instruct's memory and quant figures.

ByteDance's open 36B with a 512K context. Apache-2.0.

Sources

Last validated 2026-08-03. Memory figures are estimates. See methodology.