Four startups betting the next wave of AI runs on something better than transformers
The neural network design powering every major chatbot is hitting real limits. A small group of companies thinks it has found a way around them.
Key points
- Google researchers published the transformer neural network design in 2017, and it now underpins every major AI chatbot on the market.
- OpenAI is on track to spend $50 billion on computing in 2025, largely because transformers are expensive to run at scale.
- The International Energy Agency predicts data-centre electricity use will double by 2030.
- At least four startups are building alternative AI architectures they say are faster and cheaper, and better suited to complex tasks.
- Liquid AI, one of those companies, has logged nearly 34 million downloads for models that run on a $50 hobbyist computer.
In the summer of 2017, a team at Google published a paper called "Attention Is All You Need." It described a new type of neural network (software loosely modelled on how brain cells connect) called a transformer. Transformers proved extraordinarily good at reading and generating text, and they became the engine inside every AI chatbot you've heard of, from ChatGPT to Claude to Gemini.
Nine years on, transformers are straining under their own success.
What is the actual problem?
To understand the meaning of a sentence, a transformer compares every word against every other word. That works well for short text. For long documents it gets expensive fast: a 10,000-word document can require 50 million individual calculations. The more text you feed in, the worse it gets.
That computational hunger has a real cost. OpenAI plans to spend $50 billion on computing this year, according to company president Greg Brockman. The International Energy Agency says total electricity consumed by data centres will double by 2030.
Transformers also struggle with the tasks AI companies are now selling hardest. Reasoning models, which write notes to themselves while working through a problem, produce large amounts of extra text the transformer must then process. AI agents, software that carries out multi-step tasks independently, need to juggle information from many sources at once. Both push transformers toward their limit.
What are the alternatives?
Four companies are attacking the problem from different angles.
Subquadratic, a Miami startup, says it has built a model called SubQ that uses "sparse attention." Instead of comparing every word with every other word, sparse attention picks only the pairings that matter, cutting the number of calculations sharply. The company claims SubQ rivals mainstream large language models on search and coding tasks, though some in the industry remain sceptical. Thousands of users are reportedly on its waitlist. We first reported on Subquadratic on 10 August 2026.
Manifest AI, based in San Francisco, replaces the attention mechanism altogether. Its approach, called "power retention," gives the model a rolling summary of what it has read, dropping information that's no longer relevant as new material arrives. Think of it like a reader who keeps good notes rather than memorising every word. The company has used the technique to build PowerCoder, based on an existing open-source coding model called StarCoder, and Brumby, which it says competes with some versions of Alibaba's popular open-source Qwen model.
Liquid AI, a spinout from MIT based in Cambridge, Massachusetts, takes a hybrid approach. Its models, called LFMs (liquid foundation models), pair transformers with a different technology called liquid neural networks, inspired by the simple nervous systems of worms. The result: models small enough to run on a Raspberry Pi, a $50 hobbyist computer, yet the company says they match the performance of rivals four times their size. Liquid AI has logged nearly 34 million downloads. Mercedes is among its customers. Our story from 4 August 2026 covers the latest LFM release in detail.
| Company | Approach | Notable claim |
|---|---|---|
| Subquadratic | Sparse attention (SubQ) | Rivals top LLMs on search and coding |
| Manifest AI | Power retention (Brumby, PowerCoder) | Competes with Alibaba Qwen variants |
| Liquid AI | Hybrid LFMs | Runs on a $50 computer; 34 million downloads |
What does this mean for ordinary people?
Nothing changes in your chatbot tomorrow. Transformers are deeply embedded in products, and replacing them takes years. But the direction matters.
If these alternative approaches work at scale, AI services could become cheaper to run, which would likely mean lower costs passed to businesses and, eventually, users. Smaller, more efficient models could also run on devices you already own, keeping your data on your phone rather than a distant server. That second point is closer than it sounds: as we reported on 5 August 2026, MacPaw is already shipping an assistant built on Liquid AI that runs entirely offline on a Mac.
MIT Technology Review, which first surfaced several of these companies, calls this next generation of models "LLMs+." Whether any of today's startups are the ones who build it is an open question. The problem they're trying to solve, though, is real and the numbers behind it are not going away.
Common questions
Do I need to do anything differently as an AI user right now?
No. These changes are happening at the engineering level, inside companies building AI tools. The products you use look the same from the outside.
Will these alternatives definitely replace transformers?
Not necessarily. Transformers have survived predictions of their replacement before. Each of these startups still has to prove its approach scales to the size of models that power major products, a significant technical and financial challenge.
