Google DeepMind's SL2T model brings American Sign Language dictation to Pixel phones

Key points

• Google DeepMind has launched SL2T, a sign-language-to-text model that turns American Sign Language into written English on a phone.
• SL2T ships first on the Pixel 11 inside Gboard, Google's keyboard app, and Live Transcribe, its captioning tool.
• The model was trained on more than 100,000 hours of signing across more than 50 sign languages, with roughly a quarter of that in ASL.
• On the FLEURS-ASL benchmark, SL2T scored 70 BLEURT zero-shot, which the team says is well above any previously reported figure.
• To protect privacy, the phone sends only body-point coordinates to the server, not camera video, and discards the original footage.

For decades, voice dictation has let hearing people talk to their phones instead of typing. Deaf users have mostly been left out of that convenience.

Google DeepMind wants to change that. The lab has released SL2T, short for sign-language-to-text, a model that watches someone sign and produces written text in another language. It's now built into two apps on the Pixel 11: Gboard, the keyboard, and Live Transcribe, which shows spoken words as captions. We previewed what Google was planning to announce the day before the Pixel 11 launch.

The first version handles American Sign Language into English. More devices and more languages are promised.

What can a user actually do with it?

Sign anywhere they'd normally type. That's the pitch, and it's a concrete one.

Gboard lets a Deaf user sign to search the web, write a message, or ask Gemini, Google's chatbot, to execute a task. In Live Transcribe, they can sign a reply during a face-to-face conversation instead of thumbing out text. Google's testers said signing in ASL felt faster and more natural than typing in English.

That matters because sign languages aren't English rendered with the hands. They're full languages with their own grammar, built from simultaneous movements of the hands, arms, face and torso.

How does the model work?

SL2T does two hard jobs at once: it watches the body carefully, then translates between two different languages.

An on-device tool called MediaPipe, a Google body-tracking library, traces points on the signer, things like knuckles and facial landmarks. Only those coordinates travel to Google's servers. The camera video stays on the phone and is thrown away. That design choice is the privacy story, and it's a meaningful one.

The server-side model then translates the moving points straight into English text. Older systems often passed through an intermediate written shorthand called "gloss." SL2T skips that step, which Google says avoids capping the vocabulary and lets quality improve as more data comes in.

Training used over 100,000 hours of signing across more than 50 sign languages, roughly a quarter of it ASL. Training on many languages together helped the model learn structures shared across them.

How good is it, really?

Good, with honest limits. On FLEURS-ASL, a public test set for ASL-to-English translation, SL2T scored 70 BLEURT without any task-specific fine-tuning. BLEURT is an automated quality score for translations. Google calls that figure well ahead of prior results.

Benchmarks aren't the whole story. The team lists real failure modes: rare signs and fast fingerspelling. One documented example turned "prey" into "grey." Passive sentences and tense when context is thin also trip it up. Separately, the engineers tackled streaming delay, hallucinated text when nobody is signing, one-handed signing while holding the phone, and fair performance for the roughly 10% of signers who are left-handed.

Who shaped it?

Deaf collaborators, from the start. The project was conceived by Sam Sepah, a Deaf engineer at Google, and Deaf partners took part in data collection and impact review.

Google also set up an AI Sign Language Advisory Committee with Deaf organisations and experts, and published a joint impact report alongside the SL2T 1.0 release in Gboard and Live Transcribe.

What should readers take from this?

If you use ASL and own a Pixel 11, sign dictation is now a built-in option in the keyboard and in Live Transcribe. Rough edges on rare or fast signs are real, and the language list will grow over time.

This is one of the first times a sign language AI has left the research lab and landed in a shipping consumer product. The benchmark score is genuinely strong. What's being slightly oversold is readiness for the full range of signing styles: the failure modes the team published honestly are the ones to watch as the rollout widens.