ByteDance Is Building One of the Biggest AI Models Ever Made

Key points

• ByteDance is training an AI model with as many as 10 trillion parameters, according to three people with knowledge of the project.
• That figure is three times larger than Kimi K3, the biggest Chinese AI model released publicly to date.
• The model is in the pre-training stage, a process that typically takes three to six months before any public release.
• The final parameter count has not been locked in and will only be confirmed at a later stage of development.
• If completed, the model would approach the scale of Anthropic's most advanced AI systems.

ByeDance, the Chinese company behind TikTok, is quietly building what could become one of the largest artificial intelligence models in the world. Three people familiar with the project told Ars Technica that the model may reach 10 trillion parameters, a number that needs a little unpacking.

A parameter is a tiny internal setting inside an AI model. The more parameters a model has, the more patterns it can learn from data, and generally the more capable it becomes at tasks like writing, reasoning and answering questions. GPT-4, the model that powers ChatGPT, is widely estimated to have around 1.8 trillion parameters. Ten trillion would be a very different beast.

For comparison, Moonshot's Kimi K3, currently the largest AI model released by any Chinese company, has roughly 3.3 trillion parameters. ByteDance's project would be three times that size.

What stage is this at?

Right now the model is in what engineers call pre-training: the first and longest phase of building an AI, where the system reads enormous amounts of text and data to learn basic language and reasoning skills. Think of it as the AI going through school before any specialist job training begins. This phase typically takes three to six months.

After pre-training, engineers run a second phase called fine-tuning, where the model is trained on more specific tasks and refined for real-world use. Only then would ByteDance consider a public release, and only if the results look good.

Importantly, the 10 trillion figure is not final. One of the sources said the exact size will only be decided later in the process. The headline number could change.

Why does this matter to ordinary people?

Larger models tend to produce better results on hard problems: complex legal documents, medical questions, multi-step coding tasks. If ByteDance ships a model at this scale, it would give the company a serious tool to compete with US labs like Anthropic and OpenAI in the global AI market.

It also signals how fast Chinese companies are closing the gap. A year ago, the gap between US and Chinese frontier AI felt wide. Today it is narrowing fast.

For consumers, the practical upshot is more competition, which historically pushes prices down and quality up. ByteDance already operates AI products including its own chatbot, Doubao, which has tens of millions of users inside China.

None of this is a finished product yet. Training can fail, results can disappoint, and companies regularly shelve models that do not perform as hoped. But the sheer scale of the attempt tells you something about where ByteDance's ambitions are pointed.