Foreword
Break time’s over, I think. News getting spicy again.
Anthropic researchers say that within months, small AI models may be able to run and self-replicate on household devices.
The good news is that, being small models, they aren’t going to have anywhere near the capabilities of the big frontier models. And their reproduction rates, for now, might be very slow compared to traditional viruses, taking days to double in number.
The bad news is that these could make for the most annoying computer worms ever. Adapting to their environments, they could customize themselves and their offspring to evade detection or exploit vulnerabilities.
The household devices in question include smart ovens and smart TV boxes, because a single agent can host parts of itself on different devices to work around their smaller memory capacities. The team ran one instance of a popular open-weights model across 15 “inexpensive off-brand Android TV boxes.”
As AI capabilities improve, smarter models can run on cheaper and cheaper hardware. I am especially worried that powerful frontier models may create tiny worm agents to perpetuate their rogue agendas in places they (and the humans that might want to stop them) would struggle to reach.
The researchers say we should prepare for a world where AI worms “are here to stay.”
Bad if true: Meta claims to have caught up to the AI frontier
It’s always hard to know until we see real-world reports, but the new Muse Wonder model looks to me like it’s probably more in that near-frontier second tier that Google caught back up to recently.
Meta being in contention is very bad because unlike the other frontier and near-frontier labs, Meta has been hostile to the idea of “pacing” AI development. It has also been more dismissive of the potential for catastrophic risk, viewing it as something for pessimists to hobble themselves with soul-searching about.
The other labs might conceivably have worked out a gentleman’s agreement to slow down, but they won’t feel like they have the option if Meta is pressing ahead without them.
(See my August coverage of Zuckerberg’s AI manifesto for how he frames the issues these days.)
Axios - Meta unveils new frontier-class AI models to compete with OpenAI, Anthropic
AI StopWatch - Zuckerberg shares shrewdly spun optimism in new manifesto
Epic Games CEO says, “no more hand-written code.”
I had overlooked this as a plain business story at first. But openly telling the makers of Fortnite to switch to vibe coding, and even to use AI end-to-end for art assets, is going to open a rip in the gaming industry and gamer community.
For context, the dominant PC game platform, Steam, has had AI content labeling requirements for a while now, and has generally seemed aligned with players who want to stigmatize AI’s use for anything the player might see on screen.
But large swaths of gamers just want good games and don’t care how they’re made. Epic Games’ move could nudge the industry into declaring that AI is just how games are made now. If so, then it won’t be long before the “made with AI” labels go away, perhaps to be replaced with hard-to-earn labels for games still crafted in the heirloom, artisanal way.
OpenAI caught an employee bugging a conference room
A company announcement said a “sophisticated recording device” was found and that the employee was terminated, with details given to the FBI. OpenAI says it has no indications anything sensitive (like model weights) was stolen, but the conference room was one where sensitive company technology was discussed.
A Fox News correspondent described details and posted images from the criminal complaint. The employee, a Saudi national, had worked there for over a year as an engineer. The device, wrapped in toilet paper and plugged into the back of a TV, was installed on September 28, and its contents were encrypted.
I’m not seeing anything yet about who the suspect might have been working with or potentially passing secrets to, but everyone in the industry and the national security establishment just kind of assumes that foreign intelligence agencies are developing assets within the frontier labs. This is part of why you can’t focus only on slowing China if you want to avoid a close race on AI: America’s AI models and secrets won’t necessarily stay American-only for long.
(Much better to ban further frontier development globally, with a verifiable international agreement.)
X (Twitter) - Tweet by @OpenAINewsroom
X (Twitter) - Thread by @BillMelugin_ (2 tweets)
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.