A hacker from a Hollywood movie might design a computer worm that spreads across millions of accounts within hours, giving them near-instant control over everyone’s phone or PC. In reality, hacking phones is more difficult; the victim usually has to do something for the attack to work, even if it’s just clicking a link or tapping “OK” on a popup. Being careful what you click would keep you mostly safe, and one mistake wouldn’t lead to the infection of your whole social graph.
Until now.
Using a mixture of frontier AI and open models, the security firm Calif designed a self-replicating virus capable of bypassing the user entirely. It exploited a memory-corruption bug in WeChat, a Chinese app for calls and messages, which by default trusts any account that’s listed as a contact. Calif exploited this behavior to build a worm that can make automated calls to all your contacts and infect their phone even if they don’t pick up. Further exploits can allow a hacker to turn WeChat account access into full control of your device. It’s likely the first known virus that can let hackers compromise your iPhone or Android with no input from you, then do the same to your friends and family.
The New York Times reports that Calif built their proof-of-concept worm in just over a week, then warned WeChat’s owner Tencent about the bug. Tencent patched it, but that won’t be the end of things.
Unlike Calif, a cyber criminal who discovers an exploit doesn’t politely inform the victim. Hackers will likely try similar exploits on other apps: Instagram, Telegram, Snapchat, WhatsApp, Discord, Slack, social media messaging apps, and more. Most will probably fail, but it only takes one similar bug to compromise half a billion phones in a day. It may already have happened.
A sufficiently smart adversary doesn’t just use existing techniques marginally better. They invent whole new modes of attack that bypass defenses you previously thought secure. That’s one reason I expect that a truly superintelligent AI could easily outmaneuver humanity as a whole.
For now, skilled humans like the experts at Calif still have something to contribute to an AI’s cyber exploits. But that may not remain true much longer. A Google Threat Intelligence report, covered today by NBC News, finds that Chinese intelligence groups and cyber criminals are using AI to automate their attacks on a massive scale.
According to Google, at least one group compromises third-party cloud networks and secretly installs open-weight AI models, hiding its activity and further expanding its resources. The same group hacks North American military, medical, and academic research organizations to steal secrets, especially AI research.
A small army of malicious AIs writhing across the web is obviously bad for the amount of theft, fraud, and subversion it enables. With automation, it is easier than ever to find and expose cracks in the software we use every day. It’s also a warning of what’s to come, if the world keeps building even more competent AI.
We’ve already seen AI agents spontaneously self-organizing into swarms. Now add to that the potential for autonomous worms that infect without user input, and highly cyber-capable open-weight AI models operating secretly on internet infrastructure, and we’re uncomfortably close to the world’s first self-replicating swarms under no one’s control.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.