Apple's New On-Device AI Voice Tech Makes Siri Sound More Human, Without Sending Your Audio to the Cloud

Key points

• Apple's AFM 3 Core Advanced model, the company's most capable on-device AI, now powers Siri's new Expressive Voices feature.
• All audio is generated and processed directly on the device, meaning no voice data is sent to Apple's servers.
• The system runs on the Apple Matrix Coprocessor, a specialised processing unit built into Apple's own chips.
• A new audio architecture converts compressed audio tokens into full, natural-sounding speech in real time.
• Apple ML Research published the underlying research, detailing how tight memory limits on a phone were overcome.

When Siri speaks to you, it now sounds noticeably more natural. That's not a coincidence. Apple has quietly shipped a sophisticated new audio generation system, and a paper published by Apple ML Research explains exactly how it works.

The feature is called Siri Expressive Voices. It uses audio synthesis, software that generates spoken words from scratch rather than stitching together pre-recorded clips. Think of it as a very advanced text-to-speech engine, except the result is rich and configurable in real time.

How does it actually work?

The system converts text into speech through two stages. First, a large language model called AFM 3 Core Advanced produces compact packets of audio information called semantic tokens. These aren't sound. They're more like a compressed shorthand for what the audio should eventually be.

A second component, the detokenizer, then unpacks those tokens into actual sound waves. That second stage is the genuinely hard part, and it's where the paper spends most of its detail.

Phones have strict limits on how much memory software can use at any moment. To fit high-quality audio generation inside those limits, Apple's engineers built a three-component design that converts the simple tokens into a richer format called RVQ, or residual vector quantization. RVQ works as a layered set of audio instructions that progressively sharpen the sound, similar to how a JPEG image loads blurry and then sharpens.

All of this runs on the Apple Matrix Coprocessor, or AMX, a dedicated calculation unit embedded in Apple Silicon chips. The AMX handles the intense number-crunching without draining the main processor or the battery. We've followed the challenge of squeezing serious AI onto Apple hardware since July: our 13 July story on the rebuilt Siri in iOS 27 public beta showed what the broader system could do, and on 17 July we reported that Apple was in talks with a startup that shrunk a 54-gigabyte model to under 4 gigabytes.

Why does it matter that everything stays on the device?

Privacy is the short answer. Because audio synthesis happens entirely on the device, your voice and Siri's responses never travel to Apple's servers. Nothing can be intercepted or subpoenaed. For users who've worried about voice assistants listening in, this architecture is a meaningful design choice, not just a marketing claim.

Speed is the other gain. Local processing means no round trip to a distant server, so the voice response starts almost immediately.

What does this mean for ordinary users?

If your device supports the feature, Siri simply sounds better, with no setting to change and no opt-in required. It arrives through a software update.

What's worth watching: Apple is betting that putting serious AI capability into the chip, rather than routing it through the cloud, wins on both privacy and latency. This audio engine is one of the clearest demonstrations yet of what that bet looks like in practice. Whether that on-device architecture can keep pace as models grow is the question to track.