Overview
To measure tokens per second on a Mac, run llama-bench for GGUF models or mlx_lm.benchmark for MLX models on your own machine, and write down the chip, memory, quantization and context length next to the result. LLM for Mac publishes no tokens-per-second figures for any Mac, because no primary source we have read gives a method-backed number for a specific chip and model. The commands below come from the llama.cpp and mlx-lm project files, read on 2026-10-13.
How many tokens per second does a Mac get?
Unknown on this site. Searches such as “mac studio llm tokens per second”, “how many tokens per second mac mini” and “m5 ultra tokens per second” return pages with figures, but a figure is only meaningful with its model, quantization, context depth, software version and test method. Apple’s spec pages list chips, unified memory and memory bandwidth; they do not list tokens per second. This site does not convert bandwidth into speed either (see methodology). Measuring takes a few minutes and answers the question for your hardware.
What is the difference between prompt speed and generation speed?
Benchmarks report two different rates, and mixing them up is the most common way a figure misleads.
| Name | Tool label | What it times |
|---|---|---|
| Prompt processing | pp in llama-bench; prompt_tps in mlx-lm | Reading the input prompt |
| Text generation | tg in llama-bench; generation_tps in mlx-lm | Producing new tokens |
The llama-bench README defines “Prompt processing (pp): processing a prompt in batches” and “Text generation (tg): generating a sequence of tokens”, plus a combined pg test. In the llama.cpp results thread, PP is measured with a batch size of 512 and TG with a batch size of 1. A long prompt makes the first reply token wait on prompt processing, so both rates matter. If chat feels slow, see local-llm-slow-on-mac.
How do you measure tokens per second with llama.cpp on a Mac?
Build llama-bench from llama.cpp and run it against a GGUF file. The command in the llama.cpp results thread, which asks for commit c1d0e7a00, is:
git checkout c1d0e7a00
cmake -B build
cmake --build build --target llama-bench -j
./build/bin/llama-bench \
-hf ggml-org/Llama-2-7B-GGUF:F16 \
-hf ggml-org/Llama-2-7B-GGUF:Q8_0 \
-hf ggml-org/Llama-2-7B-GGUF:Q4_0 \
-p 512 -n 128 -ngl 99 --delay 30 2> /dev/null
Options, from the llama-bench README:
-pis prompt tokens (default 512).-nis generated tokens (default 128).-nglis GPU layers offloaded (default -1). The thread uses 99.-ris repetitions of each test (default 5).-dprefills the KV cache with n tokens before the test (default 0). Use it to see how speed changes at long context; background on long-context-on-mac.-opicks the output format: csv, json, jsonl, md or sql (default md).-mtakes a local model path, so you can test the exact file you run daily.--delaypauses between tests, in seconds (default 0). The thread’s command sets 30. The thread does not explain why.
The README’s example output columns are model, size, params, backend, ngl, test and t/s (tokens per second, averaged over the repetitions, with a standard deviation). The thread defines t/s as “tokens per second”.
How do you benchmark mlx-lm models?
Run mlx_lm.benchmark with a model from Hugging Face or a local directory. The mlx-lm README installs with pip install mlx-lm and mentions mlx_lm.benchmark only in passing, as accepting --prefill-step-size. The arguments are in the benchmark.py source:
mlx_lm.benchmark --model <repo or path> -p 512 -g 128 -n 5
| Argument | Meaning | Default |
|---|---|---|
--prompt-tokens, -p | Length of prompt | 512 |
--generation-tokens, -g | Length of completion | 1024 |
--num-trials, -n | Number of timing trials | 5 |
--batch-size, -b | Batch size | 1 |
--prefill-step-size | Step size for prefill processing | 2048 |
--delay | Delay between each test, seconds | 0 |
The script runs a warm-up, then prints one line per trial with prompt_tps, generation_tps, peak_memory and total_time, and a final Averages: line. Install and chat commands are on mlx-lm-on-mac.
For a single prompt, mlx_lm.generate also prints a summary, per its generate.py source: Prompt: N tokens, X tokens-per-sec, Generation: N tokens, X tokens-per-sec and Peak memory: X GB. Its --max-tokens default is 100. A 100-token run is a rough check, not a benchmark; use mlx_lm.benchmark for a comparable result. Whether MLX or GGUF is faster on your Mac is unknown until you run both; format trade-offs are on mlx-vs-gguf.
What should you record so a result is comparable?
Record these with every number, or the number cannot be compared to anyone else’s:
- Chip and memory. Apple says “About This Mac shows an item labeled Chip, followed by the name of the chip” (Apple Support), opened from the Apple menu.
- Model name, parameter count and the exact file or repo.
- Quantization (for example
Q4_0,Q8_0or an MLX bit width). - Prompt tokens, generated tokens and context depth (
-p,-n,-d, or-p,-g). - Tool and version: the llama.cpp commit or the mlx-lm version.
- Repetitions and the delay between them.
- Whether the model fit in memory. Check the fit first with
total_GB = weights_GB + kv_GB + overhead_GBfrom methodology; a model that spills to swap will report a slow figure that says nothing about the chip.
How should you read other people’s tokens-per-second numbers?
Treat any figure without the seven items above as unverified. The llama.cpp thread is a community collection of results from many devices; it is third-party data and this page copies none of it. Its instructions ask contributors to run the pinned commit and to include the F16 results, not only quantized ones. If a device is already listed with comparable results, they say there is no need to add it again.
For Mac-specific pages that explain what each chip offers before you measure, see mac-studio-llm, mac-mini-llm and m5-ultra-llm.
Questions
How many tokens per second does a Mac mini get?
Unknown on this site, because Apple’s Mac mini spec pages do not publish tokens per second and this site does not publish its own. Speed depends on the chip, memory, model, quantization and context. Run llama-bench or mlx_lm.benchmark as shown above to get the figure for your Mac mini. Hardware details are on mac-mini-llm.
What are the tokens per second for a Mac Studio LLM?
Unknown here. No primary source we read gives a method-backed figure for a specific Mac Studio and model, so none is stated. Measure yours with the commands above and record the chip and memory as listed. Hardware options are on mac-studio-llm.
How many tokens per second will an M5 Ultra get?
Unknown. This site has no measured M5 Ultra figure and does not estimate one from bandwidth. See m5-ultra-llm for what Apple lists, and measure once you have the machine.
How do you benchmark mlx-lm models?
Install with pip install mlx-lm, then run mlx_lm.benchmark --model <repo or path> -p 512 -g 128 -n 5. It does a warm-up, prints prompt_tps, generation_tps and peak_memory per trial, and ends with an averages line. The arguments are listed in the table above.
How do I benchmark a Mac mini M4 with llama.cpp?
Build llama-bench, then run it on a GGUF file with -p 512 -n 128 -ngl 99, and record the pp and tg rows. The same command works on any Apple silicon Mac. The result is yours only; this site gives no Mac mini M4 figure.
What do pp and tg mean in llama-bench?
pp is prompt processing, the rate at which the input prompt is read. tg is text generation, the rate at which new tokens are produced. The llama.cpp thread measures PP with a batch size of 512 and TG with a batch size of 1, and reports both in tokens per second.
Why does this site not publish tokens-per-second figures?
A speed number without its model, quantization, context depth, software version and method is not checkable, and this site only publishes numbers it can cite to a primary source or compute with a shown formula. Where neither exists, the page says unknown and links this method.
Sources read on 2026-10-13
- llama.cpp discussion #4167: the
llama-benchcommand and flags, the pinned commitc1d0e7a00, PP and TG batch sizes, posting instructions. No contributor results taken. - llama-bench README: option list and defaults, pp/tg/pg definitions, example output columns.
- mlx-lm README:
pip install mlx-lm,mlx_lm.generate, themlx_lm.benchmarkand--prefill-step-sizemention. - mlx_lm/benchmark.py: arguments, defaults, printed labels.
- mlx_lm/generate.py: summary print lines,
--max-tokensdefault of 100. - Mac computers with Apple silicon, Apple Support: About This Mac shows the Chip.
- methodology: fit formula used in the pre-check.