Methodology

Step 01

Weights memory

We prefer the measured on-disk size of each GGUF file from Ollama and HuggingFace repos (bartowski, unsloth). When a quant is not published, we estimate it from effective bits-per-weight:

weights_GB = params_B × bpw / 8

Effective bits-per-weight (from the llama.cpp quantize benchmark table):

• Q4_K_M
  4.89 bpw
• Q8_0
  8.5 bpw
• FP16
  16 bpw

Step 02

KV cache

Context adds memory on top of weights. The full cost is 2 × n_layers × n_kv_heads × head_dim × context × bytes. Most modern models use Grouped Query Attention. Llama 3.1 8B has 8 KV heads instead of 32 query heads, so the KV cost is about 4x smaller than a naive calculation. We approximate this with a sqrt(params) scaling calibrated to real GQA models, rather than per-model head counts, so it is an estimate; exact cost needs layer details not stored in the dataset.

Step 03

Mixture-of-Experts

MoE models (like Qwen3 30B-A3B) activate only a few billion parameters per token, so they run fast. But all experts must be held in memory, so memory tracks total parameters, not active ones. We size MoE models by total params, a point many calculators get wrong.

Step 04

Usable memory

We compare the model's need to the memory you can actually give it:

• Apple Silicon: unified memory, ~66% usable under 64 GB and ~75% at or above (Metal recommendedMaxWorkingSetSize).
• Discrete GPU: VRAM minus ~1 GB for driver and display.
• CPU-only: ~60% of RAM, leaving room for the OS and apps.

Step 05

The verdict

We pick the highest-quality quant that fits in usable memory. Yes means it fits with headroom, tight means it fits but barely, no means it does not fit even at Q4_K_M. These are estimates, always verify before a large download.

Step 06

Speed estimate (tokens/sec)

Local generation is memory-bandwidth-bound: each token reads the model's weights once. So for dense models we estimate tok/s ≈ efficiency × bandwidth ÷ weights-GB, with a device-class efficiency factor (Apple unified ~0.8, discrete GPU ~0.65, CPU and phones ~0.5) sanity-checked against published llama.cpp and MLX benchmarks. Memory bandwidth is sourced per device from Apple, NVIDIA and AMD spec sheets. It is an estimate, marked "est." wherever it appears.

We do not put a tok/s number on Mixture-of-Experts models: only the active experts are read per token, so this bound overestimates them. Their pages show the active-parameter count instead (a 30B-A3B activates ~3.3B params per token, so it generates faster than its total size suggests).

Speed also drops with context. Every token re-reads the KV cache, so the per-token traffic is weights plus KV. At a long context the KV can dwarf the weights, which is why a model page shows generation falling to a fraction of its short-context speed at 128k. The retention ratio, weights ÷ (weights + KV), is the same on any device, so we show it without one.

Step 07

Running cost, power and offload

Each device carries a sourced TDP (peak watts) and street price. From the estimated tok/s we work out the electricity to generate a million tokens: cost = (TDP ÷ 1000) × hours × $/kWh, at an assumed $0.15/kWh (near the US average). TDP is the peak draw, so this is an upper bound. The breakeven figure divides the device price by the per-million-token saving versus a hosted API assumed at $0.5/million tokens; both assumptions are stated so you can rescale them. The takeaway is usually that local hardware is a fixed cost, not a per-token one.

When a model is too big for a discrete GPU's VRAM, llama.cpp can keep some layers on the card and stream the rest from system RAM. We estimate that offloaded speed by splitting the per-token read: the resident fraction at VRAM bandwidth, the spilled fraction at a conservative ~60 GB/s system-RAM bandwidth (deliberately low so we never overstate it). It is shown only on discrete GPUs, where there is a faster tier to fall back from; Apple unified memory has none, so an oversize model there simply does not fit.

Step 08

Leaderboard scores

The leaderboard ranks models on three sourced, third-party benchmarks, never our own numbers:

• Coding is the Aider polyglot benchmark (share of 225 Exercism exercises solved across six languages), read from the Aider repo's published leaderboard data.
• Tool use is the Berkeley Function-Calling Leaderboard (BFCL) Overall Acc, the share of tasks where the model picks the right tool with the right arguments. Where a model has a native function-calling (FC) result we use that, otherwise its prompted result. It is the open, sourced proxy for how agent-ready a model is.
• Chat is LMArena (Chatbot Arena) Elo from blind human votes.

We list a model on a board only after verifying its score against the canonical leaderboard for that exact model (same family, size, variant and version), so coverage is partial by design and a number is never estimated. Scores are for the full-precision model, not a specific quant, and they drift, so we re-check them. Whole-task agent leaderboards such as Agent Arena are still led by frontier cloud models with open-weight models well behind; we cite them for context but rank on the open, sourced boards above.