text model · Nemotron · Android

Can I run Nemotron Nano 9B v2 on Samsung Galaxy S24 Ultra?

Compatibility verdict VRAM threshold engine

Yes, but tight slow on this hardware ~6 tok/s est.

Yes. Nemotron Nano 9B v2 runs on Samsung Galaxy S24 Ultra at Q4_K_M (~7.6 GB of ~8.5 GB usable).

Needs ~7.6 GB Device usable ~8.5 GB

Fits at Q4_K_M (~7.6 GB of ~8.5 GB usable) but with little headroom. Close background apps, and expect slow generation on a phone.

That figure is at a 4k context and moves about ±15% as context length changes. Samsung Galaxy S24 Ultra leaves ~0.9 GB of headroom.

Q4_K_M needed: ~7.6 GB
Usable on device: ~8.5 GB
Device memory: 12 GB
Best quant: Q4_K_M

Share on X Share on Reddit

Which quant fits

Quant ladder vs ~8.5 GB usable

Q2_K

~5.3 GB

Q3_K_M

~5.9 GB

Q4_K_M

~7.6 GB

Q5_K_M

~7.9 GB

Q6_K

~8.9 GB

Q8_0

~10.3 GB

FP16

~19.3 GB

The line marks Samsung Galaxy S24 Ultra's ~8.5 GB budget; rungs past it are too large.

Running cost · estimate

Power & economics est.

Power draw: ~8 W
Electricity / 1M tokens: ~$0.06
Pays for itself after: ~2,952M tok

At ~$0.15/kWh and the estimated ~6 tok/s, a million generated tokens costs about $0.06 in electricity. Versus a hosted API at ~$0.5 per million tokens, the ~$1,299 Samsung Galaxy S24 Ultra pays for itself after roughly 2,952 million tokens, so local hardware is mostly a fixed cost, not a per-token one. TDP is the peak draw, so this is an upper bound. Assumptions.

How to run it

On Android use PocketPal AI (Polished app, download GGUF and run offline.).

Model Nemotron

Parameters: 9B
Q4_K_M size: 6.08 GB
Q8_0 size: 8.81 GB
Context: 128k

Full Nemotron Nano 9B v2 requirements →

Device Android

Memory: 12 GB ram
Usable for weights: ~8.5 GB
Power draw: ~8 W
Best runtime: llama.cpp (PocketPal) or MLC-LLM (Adreno GPU path)

Best models for Samsung Galaxy S24 Ultra →

You could also run

Run Nemotron Nano 9B v2 on other hardware

Nvidia GeForce RTX 3060 (12GB)Nvidia GeForce RTX 4070 (12GB)Generic Android Phone (12GB RAM)iPhone 17 Pro iPhone Air Apple M2 (16GB)

FAQ

Can Samsung Galaxy S24 Ultra run Nemotron Nano 9B v2?

Yes. Nemotron Nano 9B v2 runs on Samsung Galaxy S24 Ultra at Q4_K_M (~7.6 GB of ~8.5 GB usable).

How much memory does Nemotron Nano 9B v2 need?

It is a tight fit on Samsung Galaxy S24 Ultra. At Q4_K_M the weights are ~6.08 GB; with KV cache and runtime overhead, budget ~7.6 GB at a 4k context.

What is the best tool to run Nemotron Nano 9B v2 on Android?

On Android, PocketPal AI (Polished app, download GGUF and run offline.) is the go-to option. NPU acceleration is limited and chip-specific; most apps run on CPU. Expect 1B-4B class.

Embed this

A live badge for your model card or README, updated as the data is.

Markdown

[![Nemotron Nano 9B v2 on Samsung Galaxy S24 Ultra](https://localmodel.run/badge/nemotron-nano-9b/samsung-s24-ultra.svg)](https://localmodel.run/can-i-run/nemotron-nano-9b/samsung-s24-ultra)

Sources

Weights are measured from GGUF files; KV cache and overhead are computed, so totals can vary ~15% with context and runtime. Any tok/s is a bandwidth estimate. See methodology.