Nvidia and IIT Madras Build a 2.8-Trillion-Token Dataset to Catch Up Indian Languages With English AI

Key points

• Nvidia and IIT Madras's AI4Bharat lab have produced a synthetic training corpus of 2.8 trillion tokens across all 22 languages listed in India's Constitution, covering the first languages of roughly 1.2 billion people.
• The corpus includes both native-script text for all 22 languages and romanised text (written in Latin letters) for 12 of them, giving 34 distinct language-and-script combinations.
• Nvidia's open-source NeMo Data Designer framework ran the full pipeline: translating English documents, scoring each translation, repairing low-scoring outputs, and converting a subset to Latin script.
• A repair step alone recovered 438.4 billion tokens that would otherwise have been discarded.
• Ten billion tokens from a 12-language slice of the corpus have already been used to continue training Nvidia's Nemotron 3.5 Super VL model.

For most of the world's AI tools, English is the native language. Everything else is an afterthought. That gap is especially sharp for the roughly 1.2 billion Indians whose first language is not English, and it is the gap that a new joint release from Nvidia and IIT Madras is trying to close.

The two organisations, working with IIT Madras's AI4Bharat research lab, have built a corpus (a large, structured collection of text used to train AI models) of 2.8 trillion tokens covering all 22 languages recognised in India's Constitution. A token is roughly three-quarters of an English word; 2.8 trillion of them is a very large number. The release, first detailed on Hugging Face, also ships the complete open-source pipeline that produced the data, so other teams can run the same process on their own documents.

Why does this matter for ordinary Indians?

Better training data leads directly to better AI products in those languages: more accurate translation apps, voice assistants that understand regional phrasing, and chatbots that can explain a medical form or a tax document in Hindi, Tamil or Bengali rather than forcing users into English.

Mitesh Khapra, who heads AI4Bharat and appeared on the TIME100 AI list in 2025, has described the core problem plainly: Indian language technology trails English because "we do not have enough data for Indian languages." This release is a direct answer to that, adding mathematics, code and question-answer material translated into 22 languages, content types that have historically been almost entirely English-only.

How was the data actually built?

The pipeline starts with English documents drawn from 15 subsets of Nvidia's existing Nemotron-CC-v2 web text collection. Each document is fed through Nvidia's NeMo Data Designer, an orchestration framework that handles translation, quality scoring and repair at scale.

For 12 languages, the model used was Gemma-4-31B-it, Google's large language model chosen after head-to-head comparisons on complete documents. A quality rubric borrowed from Multidimensional Quality Metrics (MQM), an established open framework for classifying translation errors by type and severity, covers meaning, fluency, terminology and formatting. Only translations scoring 75 or above out of 100 pass. Lower-scoring outputs get targeted repairs and are rescored. That repair step recovered 438.4 billion tokens already included in the final count.

For the remaining 10 languages, the pipeline switches to Indic Translate, a specialist document-level translation model built by Bodhan AI, a nonprofit centre for AI in education incubated at IIT Madras, with Gemma-4-31B-it then scoring the results.

The corpus breaks down to 2,601.0 billion native-script tokens and 224.6 billion romanised tokens, measured using the Gemma-4 tokeniser. Every output record retains a link back to its English source document, which makes the dataset auditable in a way that unlabelled web scrapes are not.

AI2Day has tracked Nvidia across 129 stories since July 2026, most often in the context of AI infrastructure. This release is a different kind of story: not chips or compute, but the raw material those chips will process. The question worth watching is whether the open pipeline actually gets picked up by the developer community and extended, or whether the 2.8 trillion tokens become a ceiling rather than a floor.