More encouraging signals from Chinese president
In World AI Conference keynote, Xi Jinping flags "loss of control" risk but gives few details
The World Artificial Intelligence Conference (WAIC) is an annual international gathering held every July in Shanghai: part-trade show and part-governance forum. This year, Xi Jinping gave the opening keynote speech; it was his first time attending WAIC.
For me, the headline news is a single phrase he used: “loss of control.” I pay close attention whenever I hear that from policymakers (as I did while reading the United Nations’ Preliminary Report a few weeks ago). Xi’s closing words called for the world to “constantly refine measures to forestall loss of control.” The term used is 失控; “loss of control” is not a Western gloss or creative interpretation, though I can’t be certain that there’s not some specific connotation that’s been lost in translation. (You can find the official English translation here.)
Xi also said that AI must remain “under human control.” Not under the state’s control, or the Party’s control. That might just be rhetoric, but the CCP is made of humans. Any world in which humans have lost control of AI is definitionally one in which the Chinese government has lost control as well. In that respect, the CCP’s incentives are aligned with everyone else’s.
This matters because the standing objection to an international treaty is that China won’t come to the table. A speech like this is evidence that China is at least willing to talk. Zvi Mowshowitz comments:
No doubt they have in mind something that would favor their position, and their initial asks will look outrageous and unacceptable to us. That is how this works. [...]
It is also possible Xi is engaging in cheap talk. [...]
The way you find out and do your best with this opportunity is: You get started now.
I don’t want to overstate the case for optimism. Xi’s language on safety is bog-standard: no compute thresholds, no evaluations, no red lines, and no verification regime. Openness is being prioritized over safety; a commitment to open-weight models that anyone can download is hard to square with keeping humans in control. You can’t recall or monitor a model whose weights have been made public.
But there are signs that the Chinese government feels that tension. This push is coming amid an internal debate over the security risks of open-weight models. The newly-announced Chinese AI model Kimi K3 is purportedly an open-weight model, but the weights won’t be public until July 27. Mowshowitz thinks that the delay is safety-motivated, but we don’t know for sure.
On balance, I think that Xi’s speech is an encouraging signal. It isn’t the exact thing that I want — an international treaty to halt the development of artificial superintelligence — but this is the kind of thing I’d expect to see on the way there.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
You can receive emails of dispatches as we write them, or subscribe to our Daily Digest for a once-a-day compilation.