Apple Teaches AI to Handle Two Languages at Once, and It Gets Better the More It Practises

Key points

• Apple ML Research published a study showing an iterative training method improves AI speech recognition for Mandarin-English code-switching.
• The system uses unlabelled audio, meaning recordings with no human-written transcripts, to teach itself over multiple training rounds.
• Code-switching, the habit of mixing two languages mid-sentence, affects hundreds of millions of speakers globally and has long tripped up voice assistants.
• The approach runs three phases: automated label generation, two-stage bilingual model training, and repeated self-improvement cycles.

If you've watched a bilingual speaker mid-conversation, you know what code-switching looks like. One sentence starts in Mandarin, finishes in English, then flips back. Perfectly natural to the speakers. A nightmare for any software trying to transcribe it.

Automatic speech recognition, the technology inside Siri and every phone dictation tool, has a specific weakness here. It needs enormous amounts of labelled training data: audio recordings paired with accurate written transcripts. Clean Mandarin data exists. Mixed Mandarin-English data is rare and expensive to produce.

What did Apple actually do?

Researchers borrowed a technique called pseudo-labeling, where the AI writes its own transcripts for unlabelled audio, then trains on those self-written transcripts. Think of it as the model marking its own homework and studying from those marked papers.

The word "iterative" is what makes it interesting. Each round, the model produces better pseudo-labels, which produce better training, which produces a stronger model that writes even better labels next time. The three phases run in sequence: generating rough labels from a large pool of raw, untranscribed audio; running a two-stage bilingual training process that first grounds the model in each language separately; then looping through repeated improvement cycles. Apple's paper applies this to Mandarin-English code-switching for the first time.

This connects to a pattern we've tracked at AI2Day. Our 20 August story on Apple ML Research's 2,000-run training experiments found the same lab pushing hard on data-scarce learning problems, and this paper reads like a natural next step.

Why should anyone outside a lab care?

Better code-switching recognition has real consequences. Voice assistants stop mishearing bilingual instructions. Transcription tools stop mangling meeting notes when a speaker slips between languages. Medical settings, customer-service calls, educational software: wherever multilingual speakers talk to machines, this gap hurts.

The method's logic also extends beyond Mandarin and English. The same three-phase loop could apply to Spanish-English mixing common across the United States, or French-Arabic mixing common across North Africa.

One honest caution: a research paper describes what worked in a controlled experiment. Products built on the finding can behave differently with background noise or fast speech.

What happens next?

Apple hasn't announced a product update tied to this research. Academic papers regularly precede shipping features by one to three years, sometimes longer, sometimes never. What the study confirms is that the gap between AI and bilingual human speech is narrowing, using audio data that costs almost nothing to collect.

Takeaway: If your work involves transcribing or analysing multilingual audio, this line of research matters more than the product release cycles suggest. The practical gap between English-only tools and genuinely bilingual ones is closing.