Apple Teaches AI to Handle Two Languages at Once, and It Gets Better the More It Practises

Key points

• Apple ML Research published a study showing an iterative training method improves AI speech recognition for Mandarin-English code-switching.
• The system uses unlabelled audio, meaning recordings with no human-written transcripts, to teach itself over multiple training rounds.
• Code-switching, the habit of mixing two languages in a single sentence, affects hundreds of millions of speakers globally and has long tripped up voice assistants.
• The approach runs three phases: automated label generation, two-stage bilingual model training, and repeated self-improvement cycles.

If you have ever watched a bilingual speaker mid-conversation, you know what code-switching looks like. One sentence starts in Mandarin, finishes in English, then flips back. It sounds natural to the speakers. It is a nightmare for the software that tries to transcribe it.

Automatic speech recognition, the technology inside Siri, Google's voice search and every phone dictation tool, has a specific weakness here. It needs enormous amounts of labelled training data: audio recordings paired with accurate written transcripts. Clean Mandarin data exists. Clean English data exists. Clean Mandarin-English mixed data is rare and expensive to produce.

What did Apple actually do?

Researchers at Apple ML Research borrowed a technique called pseudo-labeling, where the AI writes its own transcripts for unlabelled audio, then uses those self-written transcripts as training material. Think of it as the model marking its own homework and then studying from those marked papers.

The twist is the word "iterative", meaning the process repeats. Each round, the model produces better pseudo-labels. Better labels mean better training. Better training produces a stronger model, which then writes even better labels in the next round. The researchers describe three phases working in sequence: generating those first rough labels from a large pool of raw, untranscribed audio; running a two-stage bilingual training process that first grounds the model in each language separately; then looping through repeated improvement cycles.

The paper, first reported by Apple ML Research, demonstrates this approach applied to Mandarin-English code-switching for the first time.

Why should anyone outside a lab care?

Better code-switching recognition has direct, practical consequences. It means voice assistants that stop mishearing bilingual instructions. It means transcription tools that do not mangle meeting notes when a speaker slips between languages. Medical settings, customer-service calls, educational software: anywhere multilingual speakers talk to machines, this gap shows up.

The method matters beyond Mandarin and English, too. The same three-phase loop could, in principle, apply to Spanish-English mixing common across the United States, or French-Arabic mixing common across North Africa.

One honest caution: a research paper describes what worked in a controlled experiment. Products built on the finding can behave differently in the messy real world, with background noise, thick accents and fast speech.

What happens next?

Apple has not announced a product update tied to this research. Academic papers regularly precede shipping features by one to three years, sometimes longer, sometimes never. What the study does confirm is that the gap between AI and bilingual human speech is narrowing, using audio data that costs almost nothing to collect.

Takeaway: If your work involves transcribing or analysing multilingual audio, watch this space. The practical gap between English-only tools and genuinely bilingual ones is closing faster than the product release cycles suggest.