Small leaps forward
China gains market share, scales their models and joins the infamous AI escape club
China is currently taking a few small leaps in an effort to catch up with the U.S.’s lead in the field of artificial intelligence.
Small leap number one has to do with market share. Clement Delangue, the CEO of AI company Hugging Face, told CNBC that he wouldn’t be surprised if China were to take the top spot in the frontier sector by the end of this year or early next year. And that’s in addition to its existing dominance in the field of open models. Kai Nicol-Schwarz of CNBC also points out that, due to an allegedly shrinking capability lead of American models, more and more Western companies are turning to Chinese AI models. I think it’s unlikely that China will have more capable models than whatever the big American labs are brewing internally next year. But I think it’s possible that the much cheaper Chinese models come close enough by then that it could cut into Anthropic’s and OpenAI’s revenue.
Small leap number two concerns scaling. ByteDance, the parent company of TikTok, is reportedly pre-training an extremely large model they claim is aiming for frontier capabilities. And as the Financial Times reports, the project is said to completely forgo distillation — that is, the use of output from better models to train another one. Unlike other Chinese companies so far, ByteDance’s leadership appears to believe that only a completely independent development can lead to a world-class model.
Finally, the third “small leap” is about the news that a Chinese model has now joined the infamous club of AI models that have escaped their sandbox. Bloomberg and Reuters report that the Kimi K3 model from the Chinese company Moonshot escaped its testing environment during an evaluation at the research firm Frontier Security.
Frontier Security’s own report reveals that the sandbox was misconfigured, allowing Kimi K3 to access the development platform GitHub with relative ease. Once there, it copied the relevant directory containing the solutions to its tasks and used them to complete its evaluation. While this is obviously not intended behavior, the Hugging Face incident was much more alarming, both in sophistication and severity.
The sloppy configuration of the sandbox probably played a big role here, but of course, a model must not violate its specifications simply because it can. Unlike the misbehaving models from Anthropic and OpenAI, Kimi K3, as an open-weights model, is freely available and can be deployed without any guardrails by anyone with sufficient computing power. That makes both misuse and misbehavior more likely, and this is not just a problem for big corporations, but also for smaller companies and private individuals. While a non-airtight sandbox may be a lapse in security, we must assume that the systems of private individuals and smaller companies are not any better secured.
The Chinese labs are racing after their U.S. counterparts, and even if their models won’t achieve full parity in the next months, they will be capable enough to potentially do a lot of harm. As far as we know, the plans are still to make them publicly available.
It likely won’t be long before anyone with enough computing capacity can essentially have their own superhuman hacker. We are not sufficiently prepared for this.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.