Mark Zuckerberg said that Meta’s AI agent Muse was “built from the ground up for privacy and security.” I don’t doubt him, but I think that they stopped building six inches off the ground. In June, I wrote about how Meta’s customer service chatbot would hand over Instagram accounts to anyone who asked nicely — as it turned out, 20,000 accounts were accessed, but Meta rejected the notion that there was a problem with its AI agent. Now Meta has produced a security hole that was decidedly worse.
Yesterday, Mac security researcher Patrick Wardle published a proof-of-concept for a vulnerability that could allow someone to take control of Muse and, through that connection, anything Muse had permission to use on your devices: Did you give Muse permission to use your camera, your calendar, location data...? Then congratulations — anyone who took control of Muse could control those, too: access emails, make purchases, record audio on the microphone, etc. The attacker could also provide instructions to Muse, which would trust those instructions and act on them. The silver lining is that Muse only launched on September 8th — the exploit was glaring enough to be discovered quickly, but the window to exploit it was short.
(As computers and programs get more complicated and LLMs get more advanced, there’s a temptation to hand it all over to ChatGPT or to Claude. I think that this makes basic fluency with computers even more essential, though, so that you have a hope of catching when your AI suggests something really crazy.)
Furthermore, once the attacker got into Muse through one device, they had access to Muse on every device that account was connected to: laptop, cell phone, baby monitor — I don’t know why you would have a smart thermostat, but if you did, the attacker could probably control that, too. That’s probably just annoying in the case of the thermostat, but much more worrying if you’ve got some kind of AI-powered home security system.
Meta shipped a hotfix very early this morning to remove the secret setting that made all this possible. So, you’re safe, at least from this screw-up. (Unless you haven’t updated the app yet. If you have Muse, please update it.) Meta’s David Singleton said on X that because local access was required, “the practical risk to users of the Muse Mac app was therefore quite low.” (Muse is not available on PC.)
It’s true that somebody would need to get malware onto your computer before they could hijack your Muse agent, but that doesn’t mean that the risk is low. One technique for delivering that malware is common enough to have a name, “ClickFix.” (You’re on the internet, you get a popup that says there’s some kind of bug, but if you click this and paste that then you can fix the bug — and presto, you downloaded malware.) Wikipedia has an article on a major ClickFix-enabled attack that just hit the government of Berlin last month. So, if this is a risk for the government of Berlin, I think that it’s a risk for Ma and Pa Facebook User, too.
We recently covered a computer worm — recently built with AI models by the security firm Calif — that could spread through the widely popular Chinese app WeChat from phone to phone without anyone clicking anything. This was a different vulnerability, an insecure AI rather than an insecurity exploited via AI, but the reason is the same: the labs are racing ahead faster than they or anybody else can keep things safe.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
You can receive emails of dispatches as we write them, or subscribe to our Daily Digest for a once-a-day compilation.