AI Learns to Budget Its Own Words Before It Starts Writing

Key points

• Apple ML Research published the Length Value Model (LenVM), a framework that predicts how many tokens an AI will still need to generate at every single step of its output.
• The technique treats each generated token, a token being the smallest chunk of text an AI processes (roughly three-quarters of a word), as carrying a small negative cost, pushing the model to be concise.
• Current AI systems manage output length only at a coarse, whole-sequence level; LenVM works token by token, giving much finer control.
• The researchers frame length control as a value-estimation problem, borrowing a method from reinforcement learning, training that rewards or penalises an AI for its behaviour.

Every time a chatbot answers your question, it generates text one small chunk at a time. Each chunk is called a token. More tokens mean more time, more electricity, and higher costs for whoever is running the service. For users, it often just means waiting longer for an answer that could have been shorter.

Researchers have known this is a problem. Their fixes, until now, have been blunt. They might tell a model "keep your answer under 200 words" and hope for the best. The model has no real feel for how many words it has left, or how much it is "spending" with every word it adds.

LenVM changes that. The system published by Apple ML Research assigns the model a running sense of remaining generation length at every decoding step, the moment it picks each new token. Think of it like a taxi meter that ticks over with every tenth of a mile, rather than a flat fare quoted at the start of the journey.

The trick is how they teach the model to care. The researchers assigned a constant small penalty to every token generated. Because the model is trained to maximise its reward, it learns on its own that unnecessary words are costly. It becomes, in effect, self-editing.

This borrows from a branch of AI training called reinforcement learning, where a model earns points for good behaviour and loses them for bad. Here, "bad" simply means verbose.

Does shorter always mean worse?

No, and that is the point. The team's results show the model trims length on tasks that do not need long answers, while keeping quality intact on tasks that genuinely require depth. The goal is not a shorter answer at any cost; it is the right-length answer.

For ordinary users this matters in two ways. First, snappier responses in day-to-day chat tools. Second, lower running costs that providers could pass on rather than absorb.

For the AI industry it opens a path toward models that budget their own computation, rather than being manually capped by an engineer setting a hard limit.