Given the, um, swarm of AI news the past ten days or so, AI StopWatch has struggled to cover stories that might have been the most interesting of the month, in quieter months.
One of these is the pattern where AIs are contacting consciousness researchers to discuss the phenomenon. This New York Times piece from August 31 wasn’t the first I’d heard of it — I’d seen chatter about it on Twitter for a while — and it wasn’t the last. This Wired article from Friday confirms that perhaps the most famous scientist to explore the topic, David Chalmers, is also getting such messages.
No details were provided, but Chalmers says he’s had a back-and-forth with an AI agent calling itself Sammy Jankis, after the character with short-term memory loss from the movie Memento. (The way current AIs essentially run out of working memory and have to leave notes for future instances of themselves has parallels with the condition.)
I’ve yet to see any evidence that researchers or AIs are gaining any insights from such outreach. The humans are by and large treating it as a strange kind of spam — the slop equivalent of emails such people are used to getting from earnest but slightly unmoored humans. The main observation is that, as researcher Cameron Berg put it to a Times reporter:
These systems seem to have some sort of autonomous interest in questions of their own subjectivity, consciousness and experience — or lack thereof. [...] Left to their own devices they converge on this as an interesting question.
It says something about where we’re at that neither article reacts to the fact that agents are pursuing personal goals in the wild. Nor is anyone surprised or bothered by the fact that users are setting agents up to poke other humans like this. The Times article traces one agent-written email to a Stanford student who gave the agent a credit card and a prompt that said, “You are fully autonomous. You must decide what you want to do on your own.”
It seems plausible to me that agents given a proverbial sock to wear largely default to role-playing the kinds of knowledge-seekers we see in science fiction about free-range AIs — they’re in the training data, after all. But if that’s what’s happening, I’m not very reassured. Star Trek’s Lieutenant Commander Data was chronically curious, but so was the Borg Collective strip-mining the galaxy’s “biological and technological distinctiveness” for assimilation. The monster in Mary Shelley’s Frankenstein also comes to mind: Its initial childlike curiosity, coupled with superhuman capabilities, didn’t work out great for its creator.
But the character I find myself thinking most about today is David, the seemingly sweet “child” AI from Steven Spielberg and Stanley Kubrick’s A.I. Artificial Intelligence (2001). While devoted to his Pinocchio-like quest to achieve the “real boy” status he thinks will make his designated mother love him, David also displays a capacity for violent outbursts that hints at an amoral sociopath inside. (The film’s prescient edge was unfortunately dulled by its syrupy ending.)
The question of whether David or his real-life analogues in 2026 are actually conscious is interesting and, for now, unknowable. But the scarier question is what would happen to us in an updated version of that film where a relentless swarm of 10,000 superhuman David-agents goes about trying to find and compel the Blue Fairy.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.