Pass me that splewdriver?
A new type of cyber attack exploits the hallucinations of coding agents
A classic scheme of the early internet era was “typo squatting”: buying a website domain name that people might type by mistake. A page like “yagoo” or “myspage.com” could be used to serve malware or fool a user into entering their Yahoo or MySpace credentials.
An AI-era twist on this scheme is the so-called “HalluSquatting” attack, the recent subject of a Fox News column by Kurt Knutsson. This attack exploits the fact that AI coding agents sometimes hallucinate — the “hallu” part of HalluSquatting — and do so in predictable ways. Because today’s AI agents lack shared and long-term memories, it’s like every new session of Claude or ChatGPT is waking up at the start of a time loop. This means they may unknowingly stumble into some of the same mistakes they made the last time around, if put to work on a similar task. Some of these mistakes involve hallucinations, like reaching for coding tools that don’t actually exist.
As an analogy, imagine that you have a prolific assistant in your workshop who just used your bandsaw, lathe, and drill press before reaching for a splewdriver. Not finding one, they tell you, “Hey, I notice you don’t have a splewdriver. It’s the right tool for this job. Mind if I order us a splewdriver?”
You scratch your head, because you’ve never heard of a splewdriver. But your assistant is really skilled at their job, so you assume they know what they’re talking about and say, “Sure.”
What happens next? Ideally, they fail to order a splewdriver, because the tool doesn’t exist. But what if it does? What if someone made a tool called a splewdriver and listed it in a marketplace for your assistant to find? Now you’re in trouble, because that tool could be anything.
This is basically what can happen to coding agents making use of open source code repositories. They will sometimes hallucinate and try to download code libraries that shouldn’t exist. Bad guys who have studied those agents make those libraries exist. Coding agents download those libraries, installing malware or passing vulnerabilities into their users’ new code for bad guys to exploit later.
This attack can carry much less friction than in my workshop analogy. Users who tire of approving every little action can configure their agents with blanket permission. A malicious package an agent reaches for probably doesn’t have an obviously suspicious name like “splewdriver” that would cause an attentive user to perk up. And the download is probably free.
A research paper on HalluSquatting found hallucination rates as high as 85% in scenarios where agents act to download code libraries. They also found that AIs from different companies sometimes hallucinate the same non-existent tool names, which could make squatting attacks on those names especially lucrative.
It’s not clear how widespread these attacks are, but they are proven to work. For now, the only real defense is the obvious one: Don’t let your agent download software for you, and only use code libraries you know and trust.
My worry is that many of the people now getting into vibe coding lack even a minimal coding background, and so might not even know what a code library is. If the word “bandsaw” is just as alien to you as “splewdriver”, you’re probably just going to let your assistant run the shop.
There’s a larger lesson here about the difficulty of trying to oversee the work of an expert that outclasses you in a given domain. The AI companies hoping to keep superhuman AIs in check by leaving their designs to slightly less superhuman AIs are racing to be in a position where everything sounds like a splewdriver to them.
If we let them do that, we’re splewed.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.