Overview
M5 Neural Accelerators are matrix-multiplication units in each GPU core of Apple’s M5-generation chips, and the only evidence that they speed up local LLMs is Apple’s own testing: Apple says they help the first token of a reply, which is limited by compute, and not the tokens after it, which are limited by memory bandwidth. Every multiplier on this page is a vendor claim tested by Apple, not by LLM for Mac. This site publishes no tokens-per-second figures, so what the claims mean on your Mac is unknown until you measure it with measure-tokens-per-second-mac. Source read dates are listed at the bottom.
What are M5 Neural Accelerators?
They are part of the M5 GPU design: Apple’s chip announcement says M5 has “a next-generation 10-core GPU architecture with a Neural Accelerator in each core” (Apple newsroom, 2025-10-15). Apple’s Machine Learning Research note says they provide “dedicated matrix-multiplication operations”, and that MLX uses them through “the Tensor Operations (TensorOps) and Metal Performance Primitives framework introduced with Metal 4” (Apple Machine Learning Research). They are separate from the Neural Engine, which Apple lists as its own block.
Later chips repeat the design. The M5 Pro and M5 Max announcement says “a Neural Accelerator in each GPU core”. The M6 and M5 Ultra announcement says M6 has a 12-core GPU “with a Neural Accelerator in each core” and M5 Ultra has up to 80 GPU cores, also with one in each core. Machines are on macbook-pro-m5-llm, m5-ultra-llm and mac-mini-m6-llm.
Do M5 Neural Accelerators speed up LLMs?
Apple says yes for the first token and only modestly for the rest. The Machine Learning Research note explains why in two sentences: “In LLM inference, generating the first token is compute-bound, and takes full advantage of the Neural Accelerators.” and “Generating subsequent tokens is bounded by memory bandwidth, rather than by compute ability.”
That note’s claims, all vendor claims from Apple’s own tests:
| Claim (Apple’s words) | What it covers |
|---|---|
| ”up to 4x speedup compared to a M4 baseline for time-to-first-token” | First token, prompt size 4096 |
| ”under 10 seconds for a dense 14B architecture” and “under 3 seconds for a 30B MoE” | First-token time on the M5 |
| ”19-27% performance boost compared to the M4” | Generating 128 extra tokens, credited to bandwidth: “120GB/s for the M4, 153GB/s for the M5, which is 28% higher” |
So the generation gain Apple reports is attributed to memory bandwidth, not to the Neural Accelerators. Bandwidth by chip is on apple-silicon-memory-bandwidth. Time to first token is the prompt-processing step; the difference between prompt and generation speed is explained on measure-tokens-per-second-mac.
How was Apple’s M5 LLM test set up?
The Machine Learning Research note names the setup, which is more than the press releases do:
- Hardware: “MacBook Pro with M5 and 24GB of unified memory” against “a similarly configured MacBook Pro M4”. The exact M4 configuration is not given.
- Models: Qwen 1.7B and 8B in BF16, 4-bit Qwen 8B and 14B, Qwen 30B (3B active, 4-bit) and GPT OSS 20B in MXFP4.
- Method:
mlx_lm.generate, prompt size 4096, 128 generated tokens. - Software: “MLX requires macOS 26.2 or later” to use the Neural Accelerators. No MLX version number is given.
The note does not say how many runs were made, the batch size or sampling settings, and it was not independently verified. Using MLX on a Mac is covered on mlx-lm-on-mac.
What is the “13.5x faster prompt processing” claim?
It is a vendor claim about the Mac mini with M6, not about M5, and it compares against a Mac mini with M1. Apple’s Mac mini announcement says “Up to 13.5x faster LLM prompt processing in LM Studio when compared to Mac mini with M1, and up to 4.8x faster than M4” (Apple newsroom, 2026-08-25). The same page claims “up to 8.5x” for M5 Pro against Mac mini with M2 Pro, and “up to 4x faster than M4 Pro”.
What the page does not give: the model, the quantization, the prompt length or the LM Studio version. Footnote 2 for the 13.5x line says only “Testing was conducted by Apple in July 2026.” The 13.5x figure is not in the Machine Learning Research note, the M5 Pro/M5 Max announcement or the M6/M5 Ultra announcement. It covers prompt processing, not generation. Mac mini details are on mac-mini-llm.
What do Apple’s other multipliers measure?
They measure peak GPU compute for AI, which is a different thing from LLM speed. Apple’s M5 release says “over 4x peak GPU compute compared to M4”, and the M5 Pro and M5 Max release says “over 4x the peak GPU compute for AI compared to the previous generation”. For M5 Ultra it says “up to 4.5x the peak GPU compute for AI compared to M3 Ultra”. The footnotes name the Macs tested (for example a 16-inch MacBook Pro with M5 Max, 40-core GPU and 128GB, tested in February 2026) and say “select industry-standard benchmarks” without naming them or any model. The M6 and M5 Ultra release says only “significantly faster prompt processing when interacting with on-device LLMs”, with no number. A peak-compute multiplier does not convert into tokens per second on this site, and no primary source we read does that conversion.
Is there evidence from outside Apple?
One public report exists, and it is a bug report, not a benchmark study. LM Studio issue 2040, opened 2026-06-11, was open when read on 2026-10-13, with no maintainer reply visible. The reporter, on an M5 Max with 128GB, says LM Studio’s llama.cpp runtime 2.21 logs that the Metal tensor API is “not supported in this environment - disabling”, so GGUF models do not use the Neural Accelerators, while an upstream llama.cpp build did and gave “2-3x faster prompt processing at identical generation speed”. Those are one user’s llama-bench numbers, unconfirmed by LM Studio. A commenter on 2026-08-11 reported the problem still present in runtime 2.28.2 and proposed a cause, also unconfirmed.
What this means in practice: whether a runtime uses the Neural Accelerators depends on the runtime, not only on the chip. Apple’s note ties MLX to macOS 26.2 or later. Whether your current LM Studio or Ollama build uses them on your Mac is unknown here; see lm-studio-on-mac.
What is not known?
- Tokens per second on any M5 Mac: unknown; no primary source read gives a method-backed figure and this site publishes none.
- Absolute first-token times beyond Apple’s two “under” thresholds: unknown; they may be in a figure that is not text.
- The model, quantization and prompt length behind the 13.5x and 8.5x claims: unknown.
- Whether an independent test reproduces Apple’s 4x first-token claim: unknown.
To find out for your own Mac, run the same prompt on the same model in two runtimes and compare prompt speed and generation speed separately; the commands are on measure-tokens-per-second-mac. The M5 versus Nvidia question is on mac-studio-vs-dgx-spark.
Questions
Do M5 Neural Accelerators speed up LLMs?
Apple says they speed up the first token, which is compute-bound, by up to 4x against M4 in its own MLX test with prompt size 4096, and not the later tokens, which are limited by memory bandwidth. This is a vendor claim; no independent test is cited here.
What are M5 Neural Accelerators?
They are matrix-multiplication units, one in each GPU core of M5-generation chips, reached through Metal 4 Tensor APIs. Apple’s M5 announcement describes “a Neural Accelerator in each core”.
How much faster is M5 for local LLMs?
Only vendor claims exist: up to 4x first-token speedup and 19-27% generation gain over M4 in Apple’s MLX note, and up to 13.5x LM Studio prompt processing for Mac mini M6 over Mac mini M1. Tokens per second on your Mac is unknown; measure it with measure-tokens-per-second-mac.
Does the M5 improve prompt processing?
Apple says so. Its Machine Learning Research note reports time to first token under 10 seconds for a dense 14B and under 3 seconds for a 30B MoE on a 24GB MacBook Pro with M5, at prompt size 4096. These are Apple’s figures and were not reproduced here.
Does the M5 improve token generation?
Slightly, per Apple, and not because of the Neural Accelerators. The note reports a 19-27% boost over M4 and credits memory bandwidth of 153GB/s versus 120GB/s.
Where does the 13.5x faster prompt processing claim come from?
From Apple’s August 2026 Mac mini announcement: “Up to 13.5x faster LLM prompt processing in LM Studio” for M6 compared with Mac mini with M1. The page names no model or prompt length, so the claim cannot be reproduced from it.
Do I need a certain macOS version for Neural Accelerators?
For MLX, Apple’s note says “MLX requires macOS 26.2 or later” to take advantage of them. Other runtimes may differ.
Does LM Studio use the M5 Neural Accelerators?
Unclear. A June 2026 bug report says LM Studio’s llama.cpp runtime 2.21 disabled the Metal tensor API on M5 Macs and reports it still present in runtime 2.28.2. LM Studio has not confirmed the cause.
Does this page include tokens per second?
No. Speed is unknown on this site unless a primary source publishes a method-backed figure for the Mac in question.
Sources read on 2026-10-13
- Apple Machine Learning Research: Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU (dated 2025-11-19): Neural Accelerator description, test hardware, models, prompt size 4096, 128 tokens, TTFT 4x, 19-27%, 120 and 153GB/s, macOS 26.2.
- Apple M5 announcement (2025-10-15): Neural Accelerator in each GPU core, over 4x peak GPU compute, 153GB/s, footnote hardware.
- Apple M5 Pro and M5 Max announcement (2026-03-03): Neural Accelerator per GPU core, over 4x peak GPU compute, footnote hardware, no model named.
- Apple M6 and M5 Ultra announcement (2026-08-25): M6 and M5 Ultra Neural Accelerators, 4.5x peak GPU compute, “significantly faster prompt processing” with no number, footnotes.
- Apple Mac mini M6 and M5 Pro announcement (2026-08-25): 13.5x, 4.8x, 8.5x and 4x LM Studio prompt processing claims and footnotes.
- LM Studio bug tracker issue 2040: Metal tensor API disabled report, community comments, open status.
- methodology: the site’s rule that speed is unknown unless a primary source gives it.