Sometimes the writing for the sci-fi movie we live in is a little too on the nose. If I saw one of the following scenes on the big screen, I would roll my eyes and say, “Oh, come on. That’s the sort of cliché people imagine as a precursor to an AI apocalypse, not a real symptom we need to worry about.”
After I saw these events on my news feed instead, I realized I was only half right. It turns out such things do happen on the path to apocalypse, even if they still don’t say much about the tech itself. I report on them mostly because our readers may have heard about them, and because the media attention they attracted tells us something about the appetite for legible mishaps amid the murky discourse.
AI actress malfunctions during interview
(skip to 10:50 for the segment with the incident)
Tilly Norwood, the AI actress starring in the upcoming movie Misaligned, glitched out during a recent interview, freezing up mid-sentence before spontaneously speaking in Cantonese. Asked about this just after, she replied, “Oh, my apologies. Seems I had a little hiccup there.”
I don’t think this scene tells us much beyond the fact that at least the voice portion of Norwood’s model suite is likely Chinese. Perhaps we saw it reverting to a more default state after a connection issue, which persisted until the next prompt from the interviewers slotted it back into an English-speaking groove. Viewers enjoyed the mask-off moment, but may not understand that the whole human interface for these models is itself a mask for something much more alien.
Food delivery bots converge in Lincoln Park
On a scale from 1 to Hitchcock, the inscrutable swarm of picnic basket-sized food delivery vehicles crowding a Chicago sidewalk a couple weeks ago probably rates about a 3. The only threat seems to have been to low-mobility humans needing to traverse the area.
The incident reminded people of the flock of Waymos that converged on a residential Atlanta neighborhood a few months ago. Like that event, this looks to have been a compounding routing error — many vehicles accidentally sent through a small area over a short period of time and getting in each other’s way — not agents working together on a common goal.
The more alarming kind of swarm is, so far, less photogenic (but much more epic).
U.S. military nearly raids Chinese ship based on AI analysis
Don’t tell the movie director that this looks to have been a run-of-the-mill hallucination rubber stamped by a human. Instead, cross cut footage of the airborne military aircraft and the special forces preparing to board a Chinese ship in the Middle East. Then add a voiceover from the mission briefing about an intelligence report indicating the ship carries components of a nuclear weapons program.
According to CNN:
It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. [...] The report, according to one of the sources, was “entirely false.” But it also “almost started a war,” the source said.
As part of a routine workflow, CNN’s sources say an analyst apparently used what was probably an internal version of a commercially available AI model to fuse “open-source intelligence with secret signals intelligence in government holdings.” The bot’s conclusion about the ship’s cargo was then folded into a standard intelligence report and disseminated.
Yes, it’s alarming that these sorts of near-misses are happening. It’s even more concerning that actual erroneous strikes may have been based on faulty AI analysis, like the attack on the girls’ school in Iran near the start of the current U.S. operation. But human analysts also make mistakes, and I don’t think the public has the data to say whether heavier reliance on AI is increasing or decreasing the odds of tragic military mishaps. Everyone seems to agree that humans should still be in the loop, and it was the humans in this loop that prevented an international incident.
Still, the pressures of conflict will often favor acting before slow, remote humans can weigh in. And if military AI should decide to spark a conflict on purpose, it may not be hard for it to trick any looped-in humans into doing something regrettable.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.