Google DeepMind's SL2T model brings American Sign Language dictation to Pixel phones

Key points

• Google DeepMind has launched SL2T, a sign-language-to-text model that turns American Sign Language into written English on a phone.
• SL2T ships first on the Pixel 11 inside Gboard, Google's keyboard app, and Live Transcribe, its captioning tool.
• The model was trained on more than 100,000 hours of signing across over 50 sign languages, with about a quarter of that in ASL.
• On the FLEURS-ASL benchmark, SL2T scored 70 BLEURT zero-shot, which the team says is well above any previously reported figure.
• To protect privacy, the phone sends only body-point coordinates to the server, not the camera video, and discards the original footage.

For decades, voice dictation has let hearing people talk to their phones instead of typing. Deaf users have mostly been left out of that convenience.

Google DeepMind wants to change that. The lab has released SL2T, short for sign-language-to-text, a model that watches someone sign and produces written text in another language. It is now built into two apps on the Pixel 11: Gboard, the keyboard, and Live Transcribe, which shows spoken words as captions.

The first version handles American Sign Language into English. More devices and more languages are promised.

What can a user actually do with it?

Sign anywhere they would normally type. That is the pitch, and it is a concrete one.

In Gboard, a Deaf user can sign to search the web, write a message, draft a document, or ask Gemini, Google's chatbot, to do a task. In Live Transcribe, they can sign a reply during a face-to-face conversation instead of thumbing out text. Google's testers said signing in ASL felt faster and more natural than typing in English.

That matters because sign languages are not English rendered with the hands. They are full languages with their own grammar, built from movements of the hands, arms, face, head and torso happening at the same time.

How does the model work?

SL2T does two hard jobs at once: it watches the body carefully, then translates between two different languages.

An on-device tool called MediaPipe Holistic tracks points on the signer, things like knuckles, elbows and facial landmarks. Only those coordinates travel to Google's servers. The camera video stays on the phone and is thrown away. That design choice is the privacy story, and it is a meaningful one.

The server-side model then translates the moving points straight into English text. Older systems often passed through an intermediate written shorthand called "gloss". SL2T skips that step, which Google says avoids capping the vocabulary and lets quality improve as more data comes in.

Training used over 100,000 hours of signing across more than 50 sign languages. Roughly a quarter was ASL. Training on many languages together, the team says, helped the model learn structures shared across them.

How good is it, really?

Good, with honest limits. On FLEURS-ASL, a public test set for ASL-to-English translation, SL2T scored 70 BLEURT without any task-specific fine-tuning. BLEURT is an automated quality score for translations. Google calls that figure well ahead of prior results.

Benchmarks are not the whole story. The team lists real failure modes: rare signs, fast fingerspelling (one example turned "prey" into "grey"), passive sentences, and tense when context is thin. The engineers also worked on issues a benchmark will not catch, including streaming delay, hallucinated text when nobody is signing, one-handed signing while holding the phone, and fair performance for the roughly 10% of signers who are left-handed.

Who shaped it?

Deaf collaborators, from the start. The project was conceived by Sam Sepah, a Deaf engineer at Google, and Deaf partners took part in data collection, user testing and impact review.

Google also set up an AI Sign Language Advisory Committee with Deaf organisations and experts, and published a joint impact report alongside the SL2T 1.0 release in Gboard and Live Transcribe.

What should readers take from this?

If you or someone you know uses ASL and owns a Pixel 11, sign dictation is now a built-in option in the keyboard and in Live Transcribe. Expect rough edges on rare or fast signs. Expect the language list to grow.

For everyone else, this is one of the first times a sign language AI has left the research lab and landed in a shipping consumer product.