Apple's AI Shortcut: How a 'Draft and Check' Trick Makes Reasoning Models Twice as Fast
Apple ML Research has built a smarter way to speed up AI thinking, one that checks meaning instead of counting exact words. It could cut the cost of running powerful AI in half.
Key points
- Apple ML Research published a technique called Arbitrage that speeds up AI reasoning models by up to 2x without sacrificing answer quality.
- The method improves on an existing speed trick called Speculative Decoding by judging whether a draft answer is good enough, not just word-for-word correct.
- Long Chain-of-Thought reasoning, the step-by-step internal monologue that helps AI solve hard problems, is particularly expensive to run, making speed gains here unusually valuable.
- The technique targets large language models, the technology behind chatbots like ChatGPT and Claude, at the inference stage, which is the moment the AI actually generates an answer for a user.
Every time you ask a powerful AI chatbot a hard question, something expensive happens behind the scenes. The model thinks through the problem step by step, a process researchers call Chain-of-Thought reasoning, before giving you an answer. All that thinking burns through computing power fast.
Apple ML Research has published a paper describing a technique called Arbitrage that could make that process significantly cheaper.
What is the problem it is solving?
The AI industry already has a speed trick called Speculative Decoding. The idea is simple: use a small, fast but less accurate model to write a rough draft of the answer, word by word. Then let the big, accurate model check that draft all at once. Checking in bulk is faster than generating from scratch.
The catch? The big model throws out words the moment they do not match its own prediction exactly, even when the meaning is perfectly fine. If the draft says "therefore" and the big model would have written "so", that counts as a failure. The draft gets rejected and work starts over. That is wasted effort.
Arbitrage fixes this by asking a smarter question. Instead of "is this word identical?" it asks "does this answer lead to a correct result?"
How does it actually work?
The system trains a small scoring model, called an advantage model, to judge whether a draft step is good enough to keep, even if the wording differs. The name Arbitrage comes from the idea of spotting value that others miss: finding draft tokens, the individual word-chunks an AI produces, that are correct in substance even if they are not a perfect match.
Because fewer drafts get thrown away, the fast small model contributes far more useful work. The big model spends less time regenerating steps from scratch.
In testing, the approach reached speeds up to twice as fast as running the large model alone, while keeping answer quality level.
Why does this matter to ordinary users?
Faster inference, meaning faster answer generation, translates directly to shorter wait times and lower running costs for the companies that build AI products. Lower costs can mean cheaper subscriptions, faster responses, or AI features reaching devices with less computing power.
Chain-of-Thought reasoning is what lets modern AI handle maths problems, legal analysis, coding and complex planning. Making it cheaper to run means those capabilities can reach more people.
Arbitrage will not change what you see on screen today. But the plumbing it improves is under almost every serious AI product you already use.
Common questions
Does this make AI answers less reliable?
No. The advantage model is trained to accept draft steps only when they lead to correct outcomes. Answer quality in Apple's tests held steady while speed increased.
Will this appear in Apple's own products?
Apple ML Research published this as academic research. There is no product announcement yet, but techniques like this routinely move from research papers into shipping products within one to two years.
