Mutually assured cooperation
China's open source AI strategy between strategic interests and security concerns
Today the New York Times reports that there are signs China is rethinking its open source AI strategy over security concerns. The leading Chinese AI corporations often use outputs from more capable AI models to train their own models, a process called “distillation.” This is a cheap way to achieve a better result than what would be possible otherwise. It also allows them to offer their models much cheaper than their American competitors.
This is the main reason Chinese models are so popular, and it has worked well for the last two years. The Chinese models have millions of enthusiastic users. But even though Chinese models lag behind, their growing capabilities worry Chinese leadership.
It seems to me like the Chinese government takes AI risk in areas like cybersecurity and biosecurity seriously. Of course there’s also the inherent regime paranoia about threats against the rule of the Chinese Communist Party (CCP). China likely worries that its citizens will use AI to evade censors, just as the West worries about AI-assisted propaganda. Still, I think Xi Jinping’s repeated warnings about humanity losing control over AI are legitimate and credible.
The first consequences of this are already becoming apparent. Kimi K3 is one of the latest Chinese open-weights models, and in some areas it matches the capabilities of the best American models. Unusually for a Chinese open model, its weights were not released until a week after Kimi K3 was made available to users.
Earlier this month, Reuters reported that the Chinese government held discussions with Chinese AI companies to explore the possibility of restricting foreign access to Chinese models.
Yuyuantantian, a Chinese blog affiliated with China’s national television broadcaster CCTV and known for launching trial balloons for the CCP’s public opinion guidance, comments on these developments as follows:
China supports openness, but that does not mean advocating the unconditional proliferation of all capabilities. In governing a model, the first questions should be what capabilities it has and what risks it might pose — not which country it comes from.
I must admit, I think that sounds reasonable. The New York Times and Bloomberg, both of which cite the post, see this primarily as a strategic reorientation of China’s open-source AI approach, and that is certainly part of it. Yes, Yuyuantantian’s post also contains a great deal of anti-Western rhetoric and a strong emphasis on AI as a strategic asset in geopolitics. The Chinese government likely took a close look at what happened with the Fable ban.
But I find a few details worth mentioning. When the post refers to “proliferation,” the Chinese original text uses 扩散 (kuòsàn). This is exactly the same term used in the context of nuclear security, such as in the Chinese translation of the Non-Proliferation Treaty — the treaty designed to prevent the uncontrolled spread of nuclear weapons and nuclear technology. This is an indication that, in Chinese discourse, the threat of AI is viewed as similar to that of nukes.
And despite the article’s propagandist rhetoric, it also contains a clear warning about the risks and a call for international cooperation:
AI risks can spread across platforms and across borders. Countries need to push for mutually compatible evaluation standards and establish mechanisms for reporting major security incidents and jointly handling vulnerabilities.
It is clear that China has different interests and values than the U.S. and the West. But it should be just as clear that mitigating the extinction risk posed by AI is a shared strategic goal. We don’t need to abandon our interests and values to cooperate on achieving this goal.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.