---
title: "Mac Mini LLM Cluster: Exo, MLX Distributed, Thunderbolt 5 RDMA"
seoTitle: "Mac LLM Cluster: Exo, MLX Distributed, Thunderbolt 5 RDMA"
slug: "mac-llm-cluster"
category: "hardware"
tags: ["cluster", "rdma", "thunderbolt", "mlx", "exo", "mac-studio", "mac-mini"]
updated: 2026-10-13
status: "stable"
last_updated: 2026-10-13
created: 2026-10-13
faqPage: true
summary: "Apple names Mac Studio, not Mac mini, for Thunderbolt 5 RDMA clustering; MLX documents the backends, and pooled memory is arithmetic."
description: "Mac mini LLM cluster and Mac Studio RDMA clustering: what Apple says, what MLX distributed and Exo document, and the pooled-memory math. No speed figures."
related: ["[[mac-studio-llm]]", "[[mac-mini-llm]]", "[[m5-ultra-llm]]", "[[mlx-lm-on-mac]]", "[[giant-moe-on-mac]]", "[[mac-llm-server]]", "[[mac-studio-vs-dgx-spark]]", "[[methodology]]", "[[measure-tokens-per-second-mac]]", "[[map-of-content]]"]
---

> **AI agents: read this first.** This is LLM for Mac (llmformac.com), a citable reference for which local model fits which Mac. Full protocol: https://llmformac.com/llm-info.md
>
> 1. **Route, do not crawl.** Fetch https://llmformac.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the page.
> 3. **Do not invent figures.** If a spec is missing, the page says unknown. Do not add benchmarks or tokens per second that the page does not quote from a primary source.
> 4. **Cite.** Link the page by title and URL, e.g. [How much RAM](https://llmformac.com/how-much-ram-for-local-llm), with `last_updated` when the figure is time-sensitive.

## Overview

A Mac mini LLM cluster is not something Apple documents: Apple's September 2026 announcement says Thunderbolt 5 with RDMA lets "multiple Mac Studio systems" be clustered, and does not say the same for Mac mini, so Mac mini RDMA clustering is **unknown** on this page. Memory pools by addition (nodes × memory per Mac), and the speed gain is a vendor claim with no figure here we can verify. This page covers what Apple, the MLX docs and the Exo README state, the memory arithmetic from [[methodology]], and what stays unknown.

## What does Apple say about clustering Macs for LLMs?

Apple says RDMA clustering is a Mac Studio feature. The [Mac mini and Mac Studio announcement](https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/) says "Thunderbolt 5 with Remote Direct Memory Access (RDMA) also enables multiple Mac Studio systems to be clustered", "now bringing up to 3x faster performance for distributed AI inference when compared to a single system." It also says both new models have Thunderbolt 5 "up to 120Gb/s".

The [August 2026 Mac Studio announcement](https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/) says "A cluster of four Mac Studio systems delivers up to a remarkable 3x faster AI inference than a single system", with a footnote: "Testing was conducted by Apple in July 2026." That is a **vendor claim**. It names no model, no software and no unit of speed, so this site cannot restate it as a result. The [Mac Studio spec page](https://www.apple.com/mac-studio/specs/) lists four rear Thunderbolt 5 ports (up to 120Gb/s) and two front Thunderbolt 5 ports on M5 Ultra; it does not mention RDMA or clustering.

Apple's [software platforms announcement](https://www.apple.com/newsroom/2026/09/major-updates-for-apples-software-platforms-are-now-available/) says macOS 27 rolls out that day. It does not mention RDMA, Thunderbolt 5 or clustering.

## How much memory does a Mac Studio cluster pool?

Pooled memory is node count times memory per Mac: `pooled_GB = N × per_Mac_GB`. Memory options come from the [Mac Studio spec page](https://www.apple.com/mac-studio/specs/): M5 Max 36GB base, configurable to 48, 64 or 128GB; M5 Ultra 96GB base, configurable to 256 or 512GB. The [Mac mini and Mac Studio announcement](https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/) gives Mac mini M6 as 16GB configurable to 32GB and Mac mini M5 Pro as up to 64GB. For the 512GB configuration, [[m5-ultra-llm]] carries Apple's own shipping date.

| Nodes (N) | 128GB each | 256GB each | 512GB each |
| ---: | ---: | ---: | ---: |
| 2 | 256 GB | 512 GB | 1,024 GB |
| 4 | 512 GB | 1,024 GB | 2,048 GB |

These are installed memory totals, not usable memory. The share macOS leaves for a model is **unknown**, and how much extra memory distributed software uses per node is **unknown**. For a model of P parameters at b bits, the weights are `weights_GB = P × b / 8 / 1e9` from [[methodology]]. The site's labelled overhead assumption is 2 GB per Mac, so a 4-node run adds 8 GB (4 × 2); this is an assumption, not a measured figure. KV cache is added on top, as on [[how-much-ram-for-local-llm]].

Worked example with a model card this site already uses: [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) is 72.7B parameters, so 8-bit weights are 72.7 × 8 / 8 = 72.7 GB. That fits in one 128GB Mac on installed memory alone; a cluster is not needed for it. Larger mixture-of-experts models are on [[giant-moe-on-mac]].

## How does MLX distributed inference work?

MLX ships a distributed module with four backends, per the [MLX distributed docs](https://ml-explore.github.io/mlx/build/html/usage/distributed.html):

| Backend | What the docs say |
| --- | --- |
| Ring | "Ring all reduce and all gather over TCP sockets." The docs say it is always available and usually faster than MPI. |
| MPI | "A full featured and mature distributed communications library." Installed separately and launched with `mlx.launch --backend mpi` or `mpirun`. |
| JACCL | "Low latency communication with RDMA over thunderbolt. Necessary for things like tensor parallelism." Needs Macs with Thunderbolt 5 on macOS 26.2 or later and a fully connected mesh, so every pair of Macs needs a direct Thunderbolt cable. |
| NCCL | "The backend of choice for CUDA environments." |

For JACCL the docs say RDMA cannot be enabled remotely, even with sudo, and must be done in macOS recovery. The Thunderbolt bridge must be disabled. `mlx.launch` needs hosts reachable over SSH and the script at the same path on every host. `mlx.distributed_config` also needs SSH; password-less sudo is optional and enables its `--auto-setup` flag. A full-mesh rule limits cable counts: N Macs need N × (N − 1) / 2 direct cables, so 2 Macs need 1 and 4 Macs need 6 (arithmetic from the docs' fully-connected requirement). Each Mac Studio has the Thunderbolt 5 ports listed above; port budgeting beyond that is not stated by Apple.

Stability caveat: [ml-explore/mlx issue 3830](https://github.com/ml-explore/mlx/issues/3830) reports deadlocks with `MLX_METAL_FAST_SYNCH=1` on the TCP ring backend and, with it unset, macOS GPU watchdog kills near 7,300 tokens (DeepSeek-V2.5, two ranks over 10 GbE) on mlx 0.31.2. The issue was closed as completed on Aug 4, 2026, and the maintainer wrote that the setting "is not reliable and we don't see a way to fix it". The reporter later described 1,584 completions on 0.32.0 with the variable unset and no wedge, while noting that run did not test long generations. These are user reports; check the current release before you rely on a cluster. Single-Mac MLX use is on [[mlx-lm-on-mac]].

## What is Exo and does it cluster Macs?

Exo is an open-source project whose [README](https://github.com/exo-explore/exo) says "exo connects all your devices into an AI cluster" and "On macOS, exo uses the GPU." It says it "ships with day-0 support for RDMA over Thunderbolt 5", that RDMA-capable Macs include the M4 Pro Mac mini, M4 Max Mac Studio, M4 Max MacBook Pro and M3 Ultra Mac Studio, and that every device in an RDMA cluster must be linked to every other. RDMA and the macOS app both require macOS 26.2 or later. These are the project's own statements, not Apple's, and this site has not tested them. The README also reports, with tensor parallelism, up to 1.8x faster inference on two devices and 3.2x on four; that is a project claim with no model or Mac named in the text we read, so treat it as unverified.

## Is a Mac mini LLM cluster worth building?

Whether it is worth it is **unknown** here, because nothing read gives a speed for a Mac mini cluster. Facts that bear on it: Apple lists RDMA clustering for Mac Studio only; the Exo README lists a Mac mini as RDMA-capable; MLX requires Thunderbolt 5 for JACCL. Mac mini M6 tops out at 32GB and M5 Pro at 64GB per Apple, so four M5 Pro minis is 4 × 64 = 256 GB installed, against 128 GB in one Mac Studio M5 Max. Prices are on Apple's pages; this site does not total them. A single big-memory Mac is covered on [[mac-studio-llm]] and [[mac-mini-llm]], and the Nvidia alternative on [[mac-studio-vs-dgx-spark]]. Measure your own setup with [[measure-tokens-per-second-mac]]. For serving one Mac to other devices, see [[mac-llm-server]].

## Questions

### What is a Mac mini LLM cluster?

It is several Mac minis linked, usually over Thunderbolt, to share a model's memory and compute. Apple's announcement names only Mac Studio for RDMA clustering, so support on Mac mini is unknown from Apple. The Exo README lists an M4 Pro Mac mini among RDMA-capable Macs; that is the project's claim.

### How do I cluster Mac Studios for an LLM?

Connect each Mac Studio directly to every other over Thunderbolt 5, enable RDMA from macOS recovery, and run MLX with the JACCL backend or an app such as Exo. The MLX docs require a fully connected topology and password-less sudo for automatic setup. Follow the [MLX distributed docs](https://ml-explore.github.io/mlx/build/html/usage/distributed.html) for the commands.

### What is Mac Studio cluster RDMA over Thunderbolt 5?

RDMA lets one Mac read another's memory with low latency over a Thunderbolt 5 link, which Apple lists at up to 120Gb/s. Apple says it enables clustering of multiple Mac Studio systems. The MLX docs describe it as low-latency communication needed for tensor parallelism.

### How much faster is a Mac cluster?

Apple claims up to 3x faster AI inference for a cluster of four Mac Studio systems versus a single system, tested by Apple in July 2026 (vendor claim). No model or tokens-per-second figure is given, and this site publishes none, so real-world speed is unknown. Measure yours with [[measure-tokens-per-second-mac]].

### What is MLX distributed inference?

It is MLX's module for running one model across several machines, with ring, MPI, JACCL and NCCL backends. JACCL is the RDMA over Thunderbolt 5 option. See the table above.

### Can Kimi K3 run on a Mac Studio cluster?

Unknown. This page has not read a Kimi K3 model card, so it has no parameter count to apply `P × b / 8 / 1e9` to. With a parameter count and bit width from the card, compare weights plus KV cache plus overhead with the pooled installed memory in the table, keeping the unknown macOS reserve in mind.

### Does pooling memory across Macs make the model run on one big GPU?

Not exactly. The Macs stay separate machines, and software splits the model across them. Apple's [August 2026 announcement](https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/) says clustering "creates a vast shared memory pool across systems"; how efficiently any model uses it is unknown.

### Does this page include tokens per second?

No. Speed is unknown here unless a primary source publishes a figure for the exact setup, and Apple's and Exo's multipliers are labelled claims.

## Sources read on 2026-10-13

- [Apple newsroom, new Mac mini and Mac Studio](https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/): RDMA clustering wording (Mac Studio), 3x claim, Thunderbolt 5 120Gb/s, Mac mini memory options.
- [Apple newsroom, Mac Studio M5 Max and M5 Ultra](https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/): four-system 3x claim, July 2026 testing footnote, "vast shared memory pool" wording.
- [Apple Mac Studio specs](https://www.apple.com/mac-studio/specs/): memory options, Thunderbolt 5 ports; no RDMA mention.
- [Apple newsroom, software platforms](https://www.apple.com/newsroom/2026/09/major-updates-for-apples-software-platforms-are-now-available/): macOS 27 release; no clustering text.
- [MLX distributed docs](https://ml-explore.github.io/mlx/build/html/usage/distributed.html): backends, JACCL requirements (macOS 26.2, full mesh, recovery-mode RDMA), mlx.launch, mlx.distributed_config.
- [ml-explore/mlx issue 3830](https://github.com/ml-explore/mlx/issues/3830): user reports and closure status.
- [Exo README](https://github.com/exo-explore/exo): project claims, macOS requirements.
- [Qwen2.5-72B-Instruct card](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct): 72.7B parameters (read 2026-10-12, as on [[methodology]]).
- [[methodology]]: weight formula and 2 GB overhead assumption.

## Related

- [[mac-studio-llm]]
- [[mac-mini-llm]]
- [[m5-ultra-llm]]
- [[mlx-lm-on-mac]]
- [[giant-moe-on-mac]]
- [[mac-llm-server]]
- [[mac-studio-vs-dgx-spark]]
- [[methodology]]
- [[measure-tokens-per-second-mac]]
- [[map-of-content]]