Overview
The best LLM for a 24GB Mac is a 4-bit build of Qwen3.8-27B or Gemma 4 26B A4B at a short context, because both list file sizes of 18 GB on Ollama and leave about 3 GB of margin by this site’s arithmetic. They stop fitting once context grows into the tens of thousands of tokens, and the share of the 24 GB that macOS lets the GPU use is unknown. This page works the margins with the formulas on methodology and does not quote any speed figure.
Is 24GB RAM enough for an LLM?
Yes for models up to roughly the 27B class at 4-bit and short context, and no for the 8-bit builds of those same models.
headroom_GB = installed_GB - (file_GB + kv_GB + overhead_GB), with overhead_GB fixed at 2 as this site’s labelled assumption. File sizes below are the sizes listed on the Ollama tag pages, not the 4.0-bit formula, because real quantized files carry more than 4.0 bits per weight.
| Build (Ollama tag) | Listed size | KV at 4096 | Total at 4096 | Headroom in 24 GB |
|---|---|---|---|---|
| qwen3.8:27b-q4_K_M | 18 GB | 0.268 GB | 20.268 GB | 3.73 GB |
| qwen3.8:27b-q8_0 | 30 GB | 0.268 GB | 32.268 GB | does not fit |
| gemma4:26b-a4b-it-q4_K_M | 18 GB | 0.923 GB | 20.923 GB | 3.08 GB |
| gemma4:26b-a4b-it-qat | 16 GB | 0.923 GB | 18.923 GB | 5.08 GB |
| gemma4:26b-a4b-it-q8_0 | 28 GB | 0.923 GB | 30.923 GB | does not fit |
| gemma4:31b-it-q4_K_M | 20 GB | 1.174 GB window-capped, 3.691 GB uncapped | 23.174 GB window-capped, 25.691 GB uncapped | 0.826 GB window-capped, does not fit uncapped |
The Gemma 31B KV comes from its config.json (60 layers: 50 sliding-window with 16 KV heads at head dimension 256, 10 full-attention with 4 KV heads at head dimension 512; the card itself does not state KV heads), as worked on best-llm-for-32gb-mac and long-context-on-mac. Whether a runtime caps the sliding layers at the 1024-token window is unknown, so both cases are shown. Treat it as a stretch on 24 GB.
Headroom is measured against installed memory, not against what macOS will actually hand the GPU. That limit is unknown here; see mac-gpu-memory-limit.
How much KV cache do Qwen3.8-27B and Gemma 4 26B A4B need?
Using kv_GB = 2 x layers x kv_heads x head_dim x context x 2 / 1e9 from methodology:
- Qwen3.8-27B. The config.json lists 64 layers in a repeating pattern of three linear-attention layers then one full-attention layer, so 16 layers are full attention, with 4 KV heads and head dimension 256. KV for those layers is 2 x 16 x 4 x 256 x context x 2 / 1e9: 0.268 GB at 4096, 2.147 GB at 32768, and 8.590 GB at 131072. The state kept by the 48 linear-attention layers is not counted and its size is unknown here.
- Gemma 4 26B A4B. The config.json lists 30 layers: 25 sliding-attention layers (8 KV heads, head dimension 256) and 5 full-attention layers (2 global KV heads, global head dimension 512). Computing every layer at full context, as an upper bound, gives 0.923 GB at 4096 and 7.382 GB at 32768. The card says sliding attention uses a 1024-token window, so a runtime that caps those layers would use less; how much Ollama or MLX saves is unknown. If they are capped, the KV is 0.294 GB at 4096 and 0.881 GB at 32768, the same window-capped basis used on gemma-on-mac and best-llm-for-48gb-mac, and the q4_K_M totals become 20.294 and 20.881 GB.
What happens at long context on 24GB?
Both models stop fitting at long context. Using the 18 GB q4_K_M files and 2 GB overhead:
| Context | Qwen3.8-27B total | Headroom | Gemma 4 26B A4B total (upper bound) | Headroom |
|---|---|---|---|---|
| 4096 | 20.268 GB | 3.73 GB | 20.923 GB | 3.08 GB |
| 32768 | 22.147 GB | 1.85 GB | 27.382 GB | does not fit |
| 131072 | 28.590 GB | does not fit | not computed | unknown |
Both cards advertise far longer windows (Qwen 262,144 tokens natively on its card, Gemma 256K), but advertised context is not usable context on 24 GB. More on this at long-context-on-mac.
Which models fit a 24GB Mac comfortably?
Smaller builds leave real room. From the same Ollama tag page, gemma4:12b-it-q4_K_M is listed at 8.0 GB and gemma4:12b-it-q8_0 at 13 GB, both under 24 GB with room for KV cache and the 2 GB overhead; the 12B KV cache is not computed here. Qwen2.5 sizes at 4.0 bits (3B to 32B) are tabulated on how-much-ram-for-local-llm. Per-family detail is on qwen-on-mac and gemma-on-mac. Picks for neighbouring sizes are on best-llm-for-16gb-mac and best-llm-for-32gb-mac.
Which 24GB Macs does Apple list?
Apple’s current spec pages list 24GB on these chips:
- Mac mini. Apple lists 24GB unified memory as a base configuration with an M5 Pro (307GB/s memory bandwidth), and 24GB options on the M6 chip, which Apple lists at 153GB/s for most configurations and 170GB/s for one. Which M6 configuration has which figure is not resolved here; see mac-mini-m6-llm.
- MacBook Pro. Apple lists 16GB, 24GB or 32GB for the 14-inch M5 (153GB/s) and 24GB as the base for M5 Pro (307GB/s) in both sizes. Apple’s page lists M5 Max with a 36GB base option and does not map a 24GB option to M5 Max.
The M4 Pro Mac mini that many readers ask about is not on the current Apple spec pages; Apple Support 121555 lists it at 24GB base, configurable to 48GB or 64GB, with 273GB/s. The M4 on that page starts at 16GB, configurable to 24GB or 32GB, at 120GB/s. The memory-fit arithmetic above depends only on the 24 GB installed, not on the chip.
Bandwidth figures are Apple’s, and this page does not convert them into a speed. Speed on your Mac is unknown until you measure it: see measure-tokens-per-second-mac.
Questions
Is 24GB RAM enough for an LLM?
Yes for 4-bit models in the 27B class at short context. Qwen3.8-27B q4_K_M (18 GB listed) totals 20.268 GB at context 4096, leaving 3.73 GB of the 24 GB, and the macOS reserve is unknown. The 8-bit builds (30 GB and 28 GB listed) do not fit.
Does Qwen3.8-27B run on 24GB of RAM?
The 18 GB q4_K_M build fits at short context by this arithmetic: 18 + 0.268 + 2 = 20.268 GB at 4096, and 22.147 GB at 32768. At 131072 it totals 28.590 GB and does not fit. Whether macOS lets the GPU use that much of the 24 GB is unknown.
Can a 24GB Mac mini run Gemma 4 26B A4B?
At short context, yes by arithmetic. The q4_K_M tag is listed at 18 GB, giving an upper-bound total of 20.923 GB at 4096; the 16 GB qat tag gives 18.923 GB. At 32768 the q4_K_M upper bound is 27.382 GB, which does not fit (20.881 GB if the sliding layers are window-capped, which would be under 24 GB). Gemma has 25.2B total and 3.8B active parameters, but all weights must still be held in memory, so the active count does not shrink the file.
What is the best local LLM for an M4 Pro Mac with 24GB?
The same two models, because fit depends on installed memory. Apple Support 121555 lists the M4 Pro at 273GB/s; speed is unknown. Start with the 18 GB q4_K_M builds at short context and check memory use yourself.
Why does a 24GB Mac struggle with a 27B model?
Because a 27B file listed at 18 GB plus KV cache and overhead leaves only about 3 GB, and long context eats that margin. Qwen3.8-27B at 32768 context leaves 1.85 GB, and the unknown macOS GPU reserve can take more. Reduce the context before reaching for a smaller model.
Is 24GB better than 16GB for local LLMs?
It lets you hold a 27B-class 4-bit file. The 18 GB q4_K_M builds exceed 16 GB outright, while 24 GB leaves about 3 GB after KV and overhead. Neither size removes the unknown macOS reserve.
How fast is a 24GB Mac with these models?
Unknown. This site publishes no tokens-per-second figures and no primary source read for this page gives one. Measure your own Mac with measure-tokens-per-second-mac.
Sources read on 2026-10-13
- Qwen3.8-27B model card and config.json: 27B parameters, 64 layers, 3:1 linear to full attention, 4 KV heads, head dimension 256, 262,144 native context, Apache-2.0.
- Ollama qwen3.8 tags: 27b q4_K_M 18GB, q8_0 30GB, bf16 56GB.
- Gemma 4 26B A4B card and config.json: 25.2B total, 3.8B active, 30 layers, layer types, KV heads, head dimensions, sliding window 1024.
- Gemma 4 31B card: 30.7B parameters, 60 layers, KV heads not stated.
- Ollama gemma4 tags: 26b-a4b q4_K_M 18GB, qat 16GB, q8_0 28GB; 12b q4_K_M 8.0GB, q8_0 13GB; 31b q4_K_M 20GB.
- Apple Mac mini specs and MacBook Pro specs: 24GB options and bandwidth figures quoted above.
- methodology: weight, KV and overhead formulas.
- Not found: a primary source for the macOS GPU memory limit on a 24GB Mac, so it stays unknown.