The M5 Max
AI Benchmark

Hardware: The One Spec That Matters

Forget CPU cores. Forget the Neural Engine. Memory bandwidth is the entire game for local AI.

Let me save you 20 minutes of reading spec sheets. Token generation during LLM inference is memory-bandwidth-bound. Every single token requires reading the entire model from memory. That means tok/s ≈ 614 GB/s ÷ model_size_GB predicts your real-world performance within 20–30%. And here's the part Apple's marketing conveniently buries: memory bandwidth is determined by GPU core count, not RAM amount. A 128 GB Mac with 32 GPU cores is slower for AI than a 64 GB Mac with 40 GPU cores. Read that again. The expensive one is slower.

The unified memory architecture is the real reason Apple Silicon dominates local AI. An NVIDIA RTX 4090 has 24 GB of VRAM. The RTX 5090 has 32 GB. Exceed those limits and your model spills to system RAM over PCIe, and performance collapses. On the M5 Max, the GPU sees all 128 GB at full bandwidth. No VRAM wall. No layer offloading. No performance cliff. This is why a laptop runs 70B models that choke a $2,000 desktop GPU. The unified memory story is the one thing Apple's marketing team actually got right.

Side-by-Side Model Comparison

Pick two models. See the truth. No marketing spin, just numbers.

RAM Tier Calculator

Stop guessing. See exactly which models fit at each RAM level (8 GB reserved for macOS).

The Benchmark Numbers (No Spin)

All models tested at 4-bit quantization on MLX, MacBook Pro M5 Max 40-core GPU, 128GB. Averaged across 3-5 passes.

I'm going to say what the press-release reviews won't: most of these models are interchangeable for everyday tasks. An 8B model at 105 tok/s and a 7B model at 111 tok/s? You will not notice the difference. Stop obsessing over single-digit tok/s differences between models in the same size class. What matters is picking the right size class for your workload and RAM.

The real story in this data is Mixture of Experts. Qwen 3 30B-A3B activates only 3 billion of its 30 billion parameters per token. That gives it the speed of a small model (127 tok/s) with the knowledge of a large one. It sits in 16 GB of RAM and it's smarter than any dense model under 27B. This is the architecture that changes everything, and it's why I keep saying: stop running 70B dense models as your daily driver. A 70B model at 12.6 tok/s is useful for batch processing. It's miserable for interactive chat. Qwen 3 30B-A3B at 127 tok/s is fast enough that you forget you're running it locally. Click any column header to sort the table.

Text Generation Models

Vision Language Models

Vision Models: The Sleeper Hit Nobody Talks About

Local image understanding at 179 tok/s. No upload to cloud servers. No privacy concerns. No API bill.

Vision Language Models are the most underrated capability of local AI on Mac. Why is everyone still uploading screenshots and documents to cloud APIs when Gemma 3 4B VLM runs locally at 179 tok/s using 2.4 GB of RAM? That's not a typo. A vision model that fits in the RAM of a smartwatch processes images faster than you can read the output. For document analysis, OCR, screenshot parsing, chart reading — this is the workflow that should make every privacy-conscious developer switch to local inference. No image of your codebase, your financial documents, or your client's data ever leaves your machine.

Here's my take: for 80% of vision tasks — summarization, classification, data extraction — Gemma 3 4B VLM running locally at 179 tok/s is better than paying for a cloud API. It's faster (under 200ms TTFT vs 500ms+ for cloud), it's free, and your data stays local. When you need serious visual reasoning, Qwen3-VL 32B at 27.3 tok/s is the best image comprehension I've tested locally, and it fits in a 64 GB Mac with room to spare. Stop paying per-image API fees for tasks a $0 local model handles.

128 GB Memory Map

Each cell represents 1 GB of unified memory. Click a model category to highlight its cells.

Speed Tier Summary

Anything above 30 tok/s feels real-time. Below that, you'll feel the wait. Choose accordingly.

Efficiency Rankings: Bang for Your Byte

Tokens per second per GB of RAM. The metric that actually tells you which models are worth their memory footprint.

Quality Evaluations

Speed without quality is worthless. Here's the truth about reasoning ability.

The speed numbers above are impressive, but they mean nothing if the model outputs garbage. I ran three standard quality benchmarks: ARC-Challenge (science reasoning), GSM8K (math), and IFEval (instruction following). The results reveal a clear quality tier that speed alone can't show you.

Agentic Benchmarks

This is where local AI falls apart. Most models can't even produce valid JSON.

The agentic results are the most damning evidence that local models aren't ready for real autonomous work. I ran 14 terminal-bench tasks — real-world shell operations in Docker containers. Parse errors dominated: most small models simply cannot produce the structured JSON output the agent loop requires. Only the simplest tasks (fix file permissions, create a file) had >30% pass rates. For context, Claude Sonnet 4.5 hits 50% on the full terminal-bench v1.0 suite. Our best local model barely cracks 25%.

RAM Tier Recommendations (The Honest Ones)

Everyone says you need 128 GB. You don't. Here's what each tier actually gets you.

The internet consensus is that you need 128 GB for local AI. That advice is wrong for most people, and it's costing them $1,000+ in unnecessary upgrades. At Q4 quantization, the best daily-driver model (Qwen 3 30B-A3B) uses 16.1 GB. Even the biggest model most people would want for interactive use (a 32B dense model) uses 17.3 GB. You need 64 GB to run 70B models, and even then, you're running them at 12.6 tok/s — which is usable, not pleasant. The 128 GB config only makes sense if you're running Qwen 3 235B, loading multiple models simultaneously, or want 70B at Q8 quality. That's a real use case, but it's not most people. Save the $1,000 and get 64 GB with the 40-core GPU.

MLX vs GGUF: One Clear Winner

This is not a close call. On M5 Max, MLX wins on speed, stability, and integration. GGUF wins on ecosystem breadth. That's it.

MLX wins. Full stop. On M5 Max hardware, Apple's MLX framework is 20–30% faster than llama.cpp for token generation. It's purpose-built for unified memory and Metal 4, and it just works. Meanwhile, Ollama — the most popular GGUF front-end — shipped with broken Metal 4 shader compilation on M5 Max as of March 2026. If you bought a brand new MacBook Pro and tried to run Ollama, you got shader errors. That's not a minor issue. That's the most popular local AI tool failing on the most popular AI hardware.

The one area where GGUF legitimately wins is model selection. There are tens of thousands of GGUF models on HuggingFace compared to thousands for MLX. If you need an obscure fine-tuned model, GGUF has it. But for the mainstream models that 95% of people run, MLX has them all through the mlx-community org on HuggingFace. The cross-platform argument for GGUF matters if you also run models on Linux or Windows. If your AI machine is a Mac — and if you're reading this, it is — use MLX.

Cloud APIs: Know When to Hold, Know When to Fold

Cloud models are still smarter. But how much smarter? And is that gap worth $25 per million output tokens?

Here's the honest truth: Claude Opus, Gemini 3.1 Pro, and GPT-5.2 are still better than any local model on the hardest reasoning tasks. Their Elo ratings (1490–1510) beat what you can run locally. But here's the question nobody asks: how often do you actually need that level of reasoning? For coding assistance, writing, summarization, data extraction, and general Q&A — which is 90% of daily AI use — a local 30B MoE model handles the job. Why would you pay $25/M output tokens for Claude Opus when Qwen 3 30B-A3B answers your question in 200ms for free? Cloud APIs make sense for the 10% of tasks that require frontier intelligence. They're a bad deal for everything else.

Cost Break-Even: The Math Nobody Does

MacBook Pro M5 Max 128GB at ~$4,999 vs blended Sonnet-tier API pricing (~$9/M tokens).

Most people never do this math. They just keep swiping their credit card for API tokens because the per-query cost feels small. But it adds up. At Sonnet-tier blended pricing (~$9/M tokens), generating 100K tokens per day costs $27/month. That's $324/year. A MacBook Pro M5 Max at $4,999 pays for itself in 15 months at that rate. And after break-even? Every single token is free. Forever. If you're a developer who generates 500K+ tokens per day (and many do, between coding assistance and document processing), the Mac pays for itself in three months. Three months. After that, you're printing free tokens while cloud users are still paying per request.

How I Tested (And Why You Should Care)

No vendor sponsorship. No review units. No cherry-picked results. Here's exactly what I did.

I'm going to tell you something most benchmark articles won't: half the numbers you see online are garbage. People run a single inference pass, screenshot the output, and call it a benchmark. That's not data. That's an anecdote. Every number in this guide comes from 3 to 5 passes per model, run in isolated subprocesses on a machine I bought at full retail price. No thermal throttling cheats. No fresh-boot single-run maximums. Real sustained performance.

The test rig: MacBook Pro 16-inch, M5 Max 40-core GPU, 128 GB unified memory, macOS 16.x. Framework: MLX 0.31.1 with mlx-lm 0.31.1. All models at 4-bit quantization. Four standardized prompts covering Q&A, reasoning, coding, and structured output. Each run in its own subprocess to prevent memory contamination. Metrics: average generation tok/s, time to first token, peak RSS memory usage. Standard laptop cooling, no external fans or cooling pads. Quality benchmarks (Elo, MMLU-Pro, HumanEval) come from public leaderboards, not my own testing — I'm benchmarking inference speed, not model intelligence.

Frequently Asked Questions

The questions people actually ask me, answered with data instead of marketing copy.

I've answered these questions hundreds of times in forums, DMs, and comment sections. The answers below are based on my benchmark data and the testing methodology described above. No hedging. No weasel words. If the data says something, I say it.