Overview
Every fit number on this site is computed with the formulas on this page. The formulas are this site’s method. They are not a single equation published by Apple. Quantization for discrete GPUs is on the sister page llm quantization explained.
Weights
Decimal gigabytes use 1 GB = 1,000,000,000 bytes.
weights_GB = params × bits / 8 / 1e9
paramsis the parameter count on the model card. For Qwen2.5-7B-Instruct the card says 7.61B (accessed 2026-10-12), soparamsis 7.61 × 10^9.bitsis the target bit width used for the example, usually 4 or 8. A 4-bit example is exactly 4.0 bits per weight. It is not the measured bits per weight of a GGUF k-quant.- The llama.cpp quantize README, accessed 2026-10-12, lists Llama 3.1 8B
Q4_K_Mat 4.8944 bits per weight and 4.58 GiB. That bits-per-weight figure is not applied to Qwen models. GiB on that table is the README’s unit. This page’s formula emits decimal GB. The two are not converted into each other.
Worked 4-bit weights for Qwen2.5 Instruct cards accessed 2026-10-12:
| Model | Parameters on the card | 4-bit weights (decimal GB) | 8-bit weights (decimal GB) |
|---|---|---|---|
| Qwen2.5-3B-Instruct | 3.09B | 1.545 | 3.09 |
| Qwen2.5-7B-Instruct | 7.61B | 3.805 | 7.61 |
| Qwen2.5-14B-Instruct | 14.7B | 7.35 | 14.7 |
| Qwen2.5-32B-Instruct | 32.5B | 16.25 | 32.5 |
| Qwen2.5-72B-Instruct | 72.7B | 36.35 | 72.7 |
8-bit weights equal the parameter count in decimal GB because bits / 8 is 1.
Mistral-7B-Instruct-v0.3 is named as a 7B model. The card excerpt read on 2026-10-12 does not give a more precise parameter count, so this site does not compute a weight size for it.
Llama 3.1 8B and 70B cards on Hugging Face were gated on 2026-10-12. This site does not invent a parameter digit for them. Published file sizes are quoted from the llama.cpp quantize README: Llama 3.1 8B original 32.1 GB and Q4_K_M 4.9 GB; 70B original 280.9 GB and Q4_K_M 43.1 GB. The same README’s 8B table lists Q4_K_M at 4.58 GiB and Q8_0 at 7.95 GiB. Speed columns in that README are not repeated.
KV cache
kv_bytes = 2 × layers × kv_heads × head_dim × context × bytes_per_value
kv_GB = kv_bytes / 1e9
The leading 2 is the key cache plus the value cache. layers, kv_heads, and hidden_size come from each model’s config.json, accessed 2026-10-12. Those files do not contain a head_dim field. This site sets head_dim = hidden_size / num_attention_heads. For every Qwen2.5 Instruct config checked, that division is 128.
bytes_per_value is assumed to be 2 (fp16 or bf16). That is an assumption, not a measurement from the runtime.
At context 4096 the KV totals are:
| Model | Layers | KV heads | KV at 4096 (decimal GB) |
|---|---|---|---|
| Qwen2.5-3B-Instruct | 36 | 2 | 0.151 |
| Qwen2.5-7B-Instruct | 28 | 4 | 0.235 |
| Qwen2.5-14B-Instruct | 48 | 8 | 0.805 |
| Qwen2.5-32B-Instruct | 64 | 8 | 1.074 |
| Qwen2.5-72B-Instruct | 80 | 8 | 1.342 |
Context 32768 is eight times 4096, so the KV column scales by 8. The 3B card says full context 32,768 tokens. The 7B, 14B, 32B, and 72B cards say full context 131,072 tokens and also say the current config.json is set for 32,768 tokens, with rope_scaling added only when a longer context is required. KV examples on this site use 4096 and 32768 because 32768 is max_position_embeddings in those config files. A 131,072-token KV total is not computed here.
Overhead and the macOS reserve
total_GB = weights_GB + kv_GB + overhead_GB
overhead_GB is assumed to be 2. That is an assumption in the range the research brief allowed (1 to 2 GB). It is not a measurement.
The share of unified memory that macOS keeps for the system is unknown. No percentage is used. A row “fits” on other pages only when total_GB is below the installed unified memory on the Apple spec page. That comparison does not subtract the unknown reserve. A total that is close to the installed figure can still fail to load.
What this method does not say
It does not predict tokens per second. It does not rank models by quality. It does not set a price. Apple’s memory bandwidth figures are quoted on which-mac-for-local-llm and are not turned into a speed.