China Trains 1.6‑Trillion Parameter Model on Domestic Accelerators

In early 2026 researchers announced that a large Chinese‑language language model has finished pretraining without using any Nvidia GPUs. The effort employed roughly 50 000 domestically produced accelerator chips across data centers in China, marking the largest scale of domestic hardware used for such a project. The resulting model contains about 1.6 trillion trainable weights, surpassing many Western counterparts that typically cap near one trillion. This milestone demonstrates that China can now build and run massive LLMs entirely with locally produced components.¹

The decision to avoid Nvidia equipment was driven partly by cost considerations. Switching to domestically manufactured silicon reduces dependence on expensive foreign hardware and sidesteps export controls tightening access to advanced chip designs from the United States since late 2023. Similar efforts by Google to train TPU-based models have been successful but are not yet as widely available or cost effective at this scale currently (2024).

With 50 000 accelerators spread over dozens of racks, engineers implemented a novel parallelization scheme that may inspire new techniques in machine-learning frameworks such as PyTorch or TensorFlow.

From a linguistic perspective, the model covers multiple Chinese language variants, indicating potential to handle broader multilingual tasks beyond just Mandarin.

By proving high-performance chips can be produced outside the US, China may reduce reliance on Western technology and lower entry barriers for emerging AI firms.

In sum, the 1.6‑trillion parameter model trained on domestic accelerators represents a landmark step toward self-contained AI production in China. It showcases local silicon capability and innovative software techniques that could influence how future LLMs are built globally. As other nations observe these developments, we may see reshaping of the AI hardware landscape in coming decade.