Why a 14B Draft Model Slows Down Your Mac Mini Instead of Speeding It Up

The Draft Model Assumption

Are you dropping a small draft model onto your Mac mini and expecting the same speedup you got on your NVIDIA box? Wondering why your tokens per second went down instead of up?

Speculative decoding works like this on CUDA hardware: a small, fast draft model guesses several tokens ahead, and the big target model checks those guesses in one batched pass. When the guesses land, you skip a bunch of expensive forward passes through the large model. This is well established on NVIDIA GPUs, and plenty of local LLM guides carry that assumption straight over to Apple Silicon without checking whether it holds.

It does not hold, at least not for a 14B target model on a Mac mini. Apple Silicon runs inference through Metal, and Metal's verification step, the part where the target model checks the draft's guesses, carries much more overhead than the equivalent step on CUDA. That overhead eats the time you saved by drafting in the first place.

The assumption breaks because Apple Silicon and NVIDIA GPUs are not the same architecture, and the drafting technique was tuned and measured on the latter. Bringing over a workflow without checking the underlying hardware math is how people end up with a slower Mac mini and no idea why.

What The Numbers Actually Show

InventiveHQ ran the actual test in June 2026, and the results should end this debate for anyone still copying CUDA playbooks onto Apple hardware. The setup was an M3 Max running Qwen2.5-Coder-14B at Q4_K_M as the target model, paired against three different draft sizes: 0.5B, 1.5B, and 3B.

Every single pairing came in slower than just running the 14B model by itself. Not marginally slower. The sweep landed at 0.41x to 0.59x of baseline throughput, meaning the fastest draft configuration still ran at less than 60 percent the speed of skipping drafting entirely.

This is a direct reversal of what the same technique does on CUDA. On NVIDIA hardware, speculative decoding with small drafts against larger targets is a known throughput win, verified batching lets the GPU check dozens of guessed tokens nearly for free. On Metal, that verification step is the expensive part, not the cheap part. The larger the draft, the more guesses to verify, the more overhead stacks up before you ever see a benefit.

Notice the drafts tested were small: 0.5B, 1.5B, 3B. None of them helped. A 14B draft model, which is what this article's headline points at, only makes that overhead problem worse, because now you're paying Metal's verification tax on a much heavier set of guesses.

What Fits On Which Mac Mini

The RAM math matters more than the draft model debate at this point. On the base 16GB M4 Mac mini, Qwen3-14B at Q3_K_M barely fits. You are looking at roughly 8-12 tokens per second, and you're forced into a short context window just to keep the thing running without swapping. Q3 quantization already gives up quality to save memory, and 16GB does not leave much room for anything else once the model weights are loaded.

Step up to the 24GB M4 Pro and the picture changes completely. That config runs Q4_K_M at 20-28 tokens per second, comfortably, with headroom for a real context window instead of a cramped one. A 14B model at Q4_K_M uses roughly 8-9 GB of weights, and the working number worth remembering is to keep total usage under 60 percent of unified memory so the KV cache and the OS have room to breathe. On 24GB, that math works. On 16GB, it barely does, and barely is not a place you want to be running anything you depend on daily.

So before you touch speculative decoding at all, ask which machine you're actually on. A 16GB M4 has bigger problems than draft model overhead. A 24GB M4 Pro has the headroom to run 14B well on its own, which makes the case for adding a draft model even weaker.

Skip The Draft, Keep The RAM

The operator-level takeaway is simple. Run the 14B model alone, as your target, with nothing drafting for it. On a 24GB M4 Pro, Q4_K_M gets you 20-28 tokens per second with real headroom, and that number does not go up by adding a draft model on top of it.

The 60 percent rule is worth writing on a sticky note. 14B at Q4_K_M eats roughly 8-9 GB of weights. Keep total unified memory usage, weights plus KV cache plus whatever the OS needs, under about 60 percent of what you have installed. On 24GB, that math leaves room to breathe. On 16GB, you're already past comfortable before you load a single extra megabyte for a draft model.

That is the actual cost people miss. A draft model is not free just because it's small. Every gigabyte you hand to a draft is a gigabyte not available for KV cache or context window, and per InventiveHQ's June 2026 numbers, that gigabyte is buying you a slowdown, not a speedup, on Metal.

So skip the draft. Put the RAM you were going to spend on it toward a longer context window or a cleaner quant of the target model instead. On Apple Silicon, that RAM earns its keep. The draft model does not.