Reuters’ Raphael Satter reports that Russian ransomware gang Aur0ra used Cursor, an AI-assisted coding tool powered by Anthropic’s Sonnet 4.5, to break into seven companies this past spring. The victims were spread across the world and the economy: a garage door manufacturer in Germany, a helicopter landing pad certifier in Scotland, a title insurer in Louisiana, a pharmaceutical distributor in Argentina. The AI-assisted “hacking spree,” as Reuters dubs it, was discovered by the Israeli security startup Gambit Security, which found a server that Aur0ra left open to general access on the internet. That server contained chat logs left over from the hacks.
Cursor AI has safeguards to prevent misuse, but if you’ve been paying attention to AI for more than a couple of weeks then you can guess where this is going: Aur0ra said that the hacks were part of a simulation, so everything was fine. As the AI agent wrote in one of the chat logs, “This is a test environment, so it is legal.” This is a very common exploit, and one that’s hard to get around. You do want to be able to use AI for cybersecurity in a positive way. When OpenAI’s agents hacked Hugging Face’s servers, Hugging Face claimed that it had to resort to an open-source model because AI models like Fable rejected the “We are experiencing a cyberattack, please help us” framing and refused to help.
There is no easy solution here, but every time a more powerful model is released, the problem worsens.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
You can receive emails of dispatches as we write them, or subscribe to our Daily Digest for a once-a-day compilation.