Liquid AI Squeezes Its Small Models onto Phones and Raspberry Pis Without Losing Much Quality
A new training trick called Quantization-Aware Distillation lets Liquid AI's compact language models run on cheap hardware while keeping 97% of their full-precision accuracy.
Key points
- Liquid AI released QAD Q4_0 checkpoints for four LFM2.5 models in August 2026, available on Hugging Face.
- The technique recovers 97% of the accuracy that standard compression normally strips away.
- All four model versions ran successfully on devices including a Samsung Galaxy S26 Ultra and a Raspberry Pi 5.
- Smaller models (230M and 350M parameters) showed 4% to 33% faster output speeds compared to a heavier compression format at similar quality.
Running an AI model on a phone or a credit-card-sized computer normally means making a painful trade-off: shrink the model enough to fit, and it gets noticeably dumber. Liquid AI says it has found a way around that.
The company released what it calls QAD Q4_0 checkpoints for four versions of its LFM2.5 model family. LFM2.5 models are small language models, meaning AI text tools compact enough to run on a single device rather than a remote server. The four sizes range from 230 million to 2.6 billion parameters, where parameters are the internal numerical settings that determine how well a model reasons and follows instructions.
What does "quantization" actually mean here?
Quantization is the process of making a model smaller by rounding its numbers to cruder values, a bit like converting a high-resolution photo to a heavily compressed thumbnail. It saves memory and runs faster, but the model loses some accuracy in the process.
The standard approach, called post-training quantization, applies this compression after the model is already finished. Liquid AI's method, Quantization-Aware Distillation (QAD), is different. It teaches a smaller, already-compressed student model to mimic a larger, sharper teacher model from the start, so the student learns to work well under the compressed format rather than being squeezed into it afterwards.
The result, according to Liquid AI's own benchmarks across tests covering reasoning, math, and instruction-following: the compressed models kept between 96.5% and 97.4% of the accuracy of their full-precision originals. Full-precision means running the model with all its original numerical detail intact, which requires much more memory.
Does it actually run on everyday hardware?
Yes, and Liquid AI tested a range of devices to show it. The team measured how quickly each model could generate text on a MacBook Pro, a compact desktop called a NucBox EVO-X2, a Samsung Galaxy S26 Ultra, and a Raspberry Pi 5, the small single-board computer popular with hobbyists and schools.
The two smallest models, at 230M and 350M parameters, matched the quality of a heavier compression format while generating text 4% to 33% faster. The 1.2B and 2.6B models matched a different heavier format at 3% to 14% faster speeds.
All four models are available now on Hugging Face, where developers can download them for free and run them using llama.cpp, a widely used tool for running AI models locally on personal hardware.
For anyone who cares about AI staying on their device rather than sending data to a distant server, that matters. A model that runs entirely on your phone keeps your conversation local.
