Same AI Memory, Fewer Tokens: How a New Approach Beats a Leading Agent System at a Fraction of the Cost
Two AI systems both teach agents to learn from their own mistakes. One sends every lesson every time. The other sends only what each model can actually use. The difference shows up in the bill.
Key points
- ALTK-Evolve matched or beat the ACE agent-memory system on a 168-task benchmark while using as little as one-seventh of the computing tokens per task.
- On DeepSeek-V3.2, a strong AI model, ALTK-Evolve scored 89.3% task completion versus ACE's 80.4%, at roughly 40% of ACE's token cost.
- On a weaker model, gpt-oss-120b, accuracy was nearly identical between the two systems, but ALTK-Evolve used about 116,000 tokens per task compared to ACE's 777,000.
- Both systems avoid compressing an agent's learned lessons into a short summary, but they differ sharply on how those lessons reach the model at decision time.
- The work was shared by the ALTK-Evolve team on Hugging Face.
Imagine teaching a new employee not through a manual, but by letting them make mistakes and then showing them exactly what went wrong. That is, roughly, what both of these AI systems do. The difference is whether you hand the employee every note ever written, or just the ones relevant to today's job.
What problem are these systems solving?
AI agents, software that carries out multi-step tasks on its own, fail in a telling way. They usually know the rules. They just apply them unreliably.
Give an AI agent a task like splitting a bill across several simulated apps, and it might call the wrong function, pull the wrong person's record, or return an answer when no answer was asked for. The agent has the knowledge. It has not yet learned how to apply it cleanly.
Both ACE (Agentic Context Engineering) and ALTK-Evolve fix this by mining the agent's own past attempts for lessons, then feeding those lessons back to the agent the next time it works. No human labels. No retraining the underlying model. Just memory, used at the moment of decision.
Where do the two systems disagree?
They disagree on delivery, and that is where the cost difference comes from.
ACE keeps one large, carefully maintained playbook and injects the whole thing into the agent's context, the block of text the model reads before acting, on every single step. That is thorough. It is also expensive.
ALTK-Evolve treats delivery as adjustable. For a strong model with plenty of headroom, it can send the full set of lessons. For a weaker model, it selects only the lessons most relevant to the current task, keeps a fixed core of high-confidence guidelines, and leaves the rest in storage. The same lessons exist in both cases. Fewer of them travel to the model.
The results on the AppWorld benchmark, a test of 168 realistic multi-app tasks, are shown below.
| Model | System | Task completion | Tokens per task |
|---|---|---|---|
| DeepSeek-V3.2 (strong) | ACE | 80.4% | 634,000 |
| DeepSeek-V3.2 (strong) | ALTK-Evolve | 89.3% | 263,000 |
| gpt-oss-120b (weaker) | ACE | 54.8% | 777,000 |
| gpt-oss-120b (weaker) | ALTK-Evolve | 56.0% | 116,000 |
Tokens are the units of text an AI model reads and writes. More tokens mean more computing time and higher cost.
Why does sending fewer lessons sometimes work better?
For harder tasks, the agent needs to pick the right lesson, not wade through all of them. On easy tasks with a weaker model, a comprehensive playbook gives a small edge, because the answer is close to the surface and more context helps. On hard tasks, that same flood of context gets in the way.
A stronger model absorbs more context without losing the thread. A weaker model can drown in it.
Both systems agree on one important principle: do not compress lessons into a short summary. A lesson that appeared in five separate tasks is meaningfully different from one that appeared once. Squashing them together loses that distinction. ALTK-Evolve tracks how many independent episodes produced each lesson. ACE keeps a helpful-versus-harmful count per item. Different labels, same underlying logic.
What happens next?
The ALTK-Evolve team say the question of exactly how many lessons to send, and how that scales across models of different strengths, is the subject of their next post. The technical library and a full report are already public for researchers who want to test the approach.
For anyone building or paying for AI agent systems, the practical point is straightforward: what a model is told at decision time shapes both accuracy and cost, and those two things do not have to trade off against each other.
Common questions
Does this mean ACE is a bad system?
No. ACE and ALTK-Evolve solve the same problem and agree on the hard parts. ACE's own efficiency story focuses on how cheaply it builds its memory store. ALTK-Evolve's efficiency gain comes from serving fewer lessons per step, which is a different axis entirely.
Does this affect ordinary users directly?
Not yet in a product you can download. The work is at the research stage, tested on a benchmark of simulated tasks. But the cost savings it points to matter for any company running AI agents at scale, and lower running costs typically translate to lower prices over time.
