The M5 Max
AI Benchmark

The Hardware: What's Actually in This Machine

Apple M5 Max, announced March 3, 2026 — TSMC 3nm, with Neural Accelerators baked into every GPU core.

Before I get into the numbers, it helps to understand why the hardware matters so much. When you're generating tokens with a language model, the bottleneck isn't compute — it's memory bandwidth. Every single token the model generates requires reading the entire set of model weights from memory. So the speed formula is pretty simple: tok/s ≈ 614 GB/s ÷ model_size_GB. That's it. That formula predicted my real-world results within about 20–30%, which I found kind of wild. The gap comes from KV cache overhead, compute ops, and framework efficiency. But bandwidth is the main event.

The other big deal is the unified memory architecture. On a regular PC with an NVIDIA GPU, you've got 24 GB of VRAM on an RTX 4090, or 32 GB on a 5090. If your model doesn't fit in VRAM, you're offloading layers to system RAM over PCIe, and that's dramatically slower. On the M5 Max, the GPU just... accesses all 128 GB directly at full bandwidth. There's no VRAM vs system RAM distinction. It's one big pool. That's what makes running a 70B model on a laptop even possible.

Compare Any Two Models

Pick two models and see how they stack up on speed, memory, and efficiency.

RAM Calculator: What Fits on Your Mac

Click a RAM tier to see which models you can actually run (I'm reserving 8 GB for macOS).

The Actual Benchmark Numbers

All models at 4-bit quantization on MLX, M5 Max 40-core GPU, 128GB. Averaged across 3-5 runs.

Alright, here's the part you probably scrolled down for. I tested every model at 4-bit quantization using MLX, ran each one 3-5 times in isolated subprocesses, and averaged the results. The table below has everything. A few things jumped out at me. First, Gemma 3 4B at 179 tok/s is generating text so fast that a full paragraph appears in under a second. Second, the Qwen 3 30B-A3B (that's a Mixture of Experts model) somehow hits 127 tok/s despite having 30 billion parameters. It only activates 3B of them per token, which gives it 8B-class speed with way more knowledge. That felt like cheating, honestly. And third, even the 70B models are usable — 12-13 tok/s is slower than reading speed, but it's not painful for longer tasks. Click column headers to sort.

Text Generation Models

Vision Language Models

Vision Models: Feeding Images to Local AI

Models that take images + text as input. Local document analysis, screenshot parsing, OCR, and visual Q&A — all without uploading anything.

I was curious whether the vision models would be noticeably slower than their text-only counterparts. Turns out, not really. Gemma 3 4B handles images at the same 179 tok/s as text. Qwen3-VL 8B was right there at 111 tok/s. The privacy angle here is what sold me — I was testing these by feeding in screenshots of my own code, photos of handwritten notes, and a few receipts. None of that data left my machine. I also tried running Pixtral (Mistral's VLM) but kept hitting an auth error during weight download, so it didn't make the final list. Something to revisit later.

For quick vision tasks — summarizing a screenshot, reading a chart, extracting text from a photo — Gemma 3 4B VLM at 179 tok/s is faster than any cloud API I've used, and it costs nothing per query. When you need the model to actually understand complex images (dense diagrams, multi-page documents), Qwen3-VL 32B at 27.3 tok/s is the strongest I tested, and it fits comfortably on a 64 GB Mac. Check the RAM guide to see which VLMs work at your tier.

128 GB Memory Map

Each cell is 1 GB of unified memory. Click a category to see what's using what.

Speed Tiers at a Glance

I grouped the models by how fast they actually feel in practice.

Efficiency Rankings: Speed per GB of RAM

Tokens per second divided by GB of memory used. Higher = more bang for your RAM buck.

Quality Evaluations

But can they actually think? I ran ARC-Challenge, GSM8K, and IFEval to find out.

Speed is great, but I also wanted to know if these models can actually reason. So I ran three standard benchmarks via lm-evaluation-harness: ARC-Challenge (grade-school science), GSM8K (math word problems with chain-of-thought), and IFEval (can the model follow precise formatting instructions). The composite score averages all three.

Agentic Benchmarks

I also tried making them do real terminal tasks. Most of them... couldn't.

Here's where things get interesting. I used terminal-bench to give each model 14 real tasks in fresh Docker containers: create files, fix git repos, install packages, write servers. The model has to figure everything out autonomously through shell commands. Green means it passed, red means it failed. Hover over the red cells to see why — most failures are "parse error" (the model couldn't even produce valid JSON) or "wrong commands" (it tried but got the shell commands wrong). For reference, Claude Sonnet 4.5 scores 50% on the full terminal-bench v1.0 suite.

My Actual Picks by RAM Tier

At Q4 quantization, model size is roughly params × 0.5 GB. I'd leave ~20% headroom for macOS and KV cache.

Look, picking the right model for your RAM is probably the most important thing you can do here. Running a model that barely fits means you've got no room for context windows, KV cache growth, or having Chrome open (and you'll have Chrome open). My picks below are based on the benchmarks I ran, with real headroom accounted for. Rule of thumb: don't use more than 80% of your memory for the model itself. Leave the rest for macOS, context, and whatever else you've got running.

MLX vs GGUF: Why I Went with MLX (and Why Ollama Broke)

Apple's MLX framework vs. the GGUF/llama.cpp ecosystem. Both work, but one works better on this hardware.

So I started this project using Ollama because, honestly, it's the easiest way to get a model running. Download, run, done. But on the M5 Max, Ollama kept throwing Metal 4 shader compilation errors. This is a known issue with Ollama 0.18.2 on M5 hardware — it uses GGUF/llama.cpp under the hood, and the Metal shaders haven't been updated for Metal 4 yet. I wasted a good two hours debugging that before switching to MLX. Once I did, everything just worked, and the performance was noticeably better anyway — roughly 20-30% faster on decode. MLX is purpose-built for Apple Silicon's unified memory and Metal 4, so that makes sense. The downside is the model ecosystem is smaller (though the mlx-community on HuggingFace has thousands of models now) and you don't get the nice GUI that Ollama or LM Studio provide.

How Do These Compare to Cloud APIs?

API pricing as of March 2026. Cloud is still better at the very top of the quality curve, but local is closer than I expected.

I think it's only fair to compare local models against the cloud options. The top cloud models (Claude Opus, Gemini 3.1 Pro, GPT-5.2) still outperform anything you can run locally on the hardest reasoning benchmarks. That's just where things are in 2026. But for the stuff I actually use AI for day-to-day — coding help, writing drafts, summarizing docs, data extraction — a locally-run 30B or 70B model handles it well. The real wins for local are privacy (nothing leaves my machine), latency (sub-200ms TTFT vs 500ms-2s for APIs), and zero marginal cost once you've bought the hardware. The cost analysis below breaks down when local starts saving you money.

When Does Local Actually Save You Money?

M5 Max 128GB at ~$4,999 vs. roughly $9/M tokens (blended Sonnet-tier pricing).

This is the question everyone asks, and the honest answer is: it depends on how much you use it. I ran the numbers assuming a $4,999 MacBook Pro and blended API costs of about $9 per million tokens (that's roughly what Sonnet-tier models cost when you average input and output). If you're doing 10K tokens a day — maybe a handful of short conversations — local doesn't make financial sense. You'd be waiting 154 years to break even. But if you're like me and you're burning through 100K+ tokens a day on coding tasks and writing, the math starts working in your favor within about 15 months. At 500K tokens/day, it's 3 months. And after that, every token is free. Electricity is maybe $5-10/month even if you're running it hard.

How I Tested All of This

Hardware, software, prompts, and what I did to keep the numbers honest.

I wanted these numbers to be reproducible, so I was pretty careful about the setup. Everything ran on a single MacBook Pro 16-inch, M5 Max with the 40-core GPU and 128 GB of unified memory, running macOS 16.x. I used MLX 0.31.1 with mlx-lm 0.31.1, and every model was tested at 4-bit quantization. No cherry-picking quantization levels to make certain models look better.

Each model got four standardized prompts: a simple Q&A, a reasoning task, a coding task, and a structured output task. I ran 3-5 passes per model and averaged the results. The important part: every benchmark run happened in an isolated subprocess. That means clean memory measurement, no contamination from previous runs, and no warm-cache advantages. I measured tok/s (generation speed), time to first token (TTFT), and peak memory (RSS). The laptop was on its standard cooling — no external fans or cooling pads. Quality numbers (Elo ratings, MMLU-Pro, HumanEval) come from public leaderboards, not my own testing. And for the record: I bought this laptop at retail. No review unit, no vendor sponsorship, no early access.

Questions I Keep Getting Asked

Stuff people asked me after reading early drafts of this post.

I shared a draft of this with a few friends and got a lot of the same questions back. I've collected them here. Most of the answers reference actual numbers from my testing, and I've tried to be specific rather than hand-wavy. If you don't see your question, the RAM Tier Guide and MLX vs GGUF section might cover it.