Overview
- The Financial Times, cited by multiple outlets, reported on Friday that ByteDance is in the pre-training phase of a model that could total about 10 trillion parameters and the company has not publicly confirmed the claim.
- Pre-training for a model of this size typically takes three to six months before fine-tuning, and industry sources say the final parameter count could change during that process.
- If the estimate holds, the model would be near industry estimates for Anthropic’s Mythos (about 8 trillion) and would be more than three times larger than Moonshot AI’s 2.8 trillion-parameter Kimi K3.
- Building and running a model at this scale would demand massive compute and energy and, according to one report, roughly 30,000 GPUs; reports also say ByteDance may use a Mixture-of-Experts architecture to reduce inference cost.
- The claim sharpens competition between Chinese and U.S. labs, may increase scrutiny of export controls and hardware access, and feeds broader policy debates about testing, containment and governance of frontier AI models.