Dispatches from Mitch
Black Ships in the harbor
Bloomberg columnist Catherine Thorbecke’s latest piece is about why Japan — and other countries — must treat AI-facilitated cyberattacks as a threat to their national security.
She makes good points about SoftBank and OpenAI’s joint pitch to sell “patching-as-a-service” being inadequate and distasteful, given how the AI industry is causing the threat it wants to protect you from. But I’m mostly here to amplify the metaphor she borrows from SoftBank’s CEO, Masayoshi Son. At a gathering of Japanese business leaders, he warned that AI cyberthreats were the “greatest crisis for Japan since the arrival of the Black Ships.”
If you’re not familiar with the Black Ships, that’s the term the Japanese used in the 16th and especially 19th centuries to describe the European ships arriving in their harbors. The most vivid incident was the arrival of U.S. Commodore Matthew Perry’s towering fleet of warships in 1853 and 1854, moving without wind and belching black smoke. There to force the Japanese into a trade agreement, Perry’s ships vividly signaled Japan’s status as an inferior technological power in danger of losing its autonomy. This inspired a difficult era of frantic political and economic transformation for the island nation.
The closest equivalent in American history is probably the Sputnik crisis of 1957, which pales in comparison. Russia putting the first artificial satellite in orbit told America that it was falling behind in rocketry; the Black Ships told Japan it was on the wrong side of an industrial revolution.
AI is much more like the Black Ships. Neither Japan nor humanity itself can assume AI is pulling only a Sputnik ahead of our present technological capabilities. Per an excerpt from an article in The Economist discussed and disputed around AI Twitter this weekend, on June 11, the head of the NSA and the Pentagon’s Cyber Command told Senator Mark Warner that Anthropic’s Mythos AI “broke into almost all of our classified systems, not in weeks, but in hours.”
Notes and clarifications have surfaced about these remarks likely being in the context of a well-resourced “red team” exercise weeks or months earlier. But the takeaway still seems to be that Mythos is much, much better at hacking than even very good human hackers, and that you really are badly outgunned in cybersecurity if you don’t have a Mythos-class model helping you shore up your defenses.
Senator Warner shared those remarks the same day Amazon found a jailbreak that might allow bad actors to use Mythos, and its more guardrailed sibling, Fable, for offensive cyber operations — a finding it went on to report to the White House. The timing looks coincidental, but the content of Warner’s remarks is seen by many as providing important context around the administration’s subsequent export control order to shut down access to Mythos and Fable pending an unspecified “license.”
Not for the first time, let me remind everyone that Mythos wasn’t trained as a specialized tool for cyberwarfare. It is a general-purpose model that by dint of being good at problem solving in general and programming in particular happens to also be great at hacking. We would be wise to worry that Mythos and its successors — rumor has it there’s already a newer Mythos in-house at Anthropic than the one the White House pulled the plug on — may be overwhelmingly better at other things, too.
It’s hard to see through all the smoke, but I think those are Black Ships in the harbor.
Chased by the reaper but “rooting for delays”
More than most, AI professor Emma Pierson knows what it means for AI to potentially cure cancer. In a piece for the Atlantic this morning, she movingly explains that her own very high cancer risk isn’t enough to make her wish for faster AI development. In fact, she’d slow AI down if she could.
She’s heard about all the AI optimism one can; she was mentored on her path to a Ph.D. by Dario Amodei, now the head of Anthropic. Amodei’s 2024 essay “Machines of Loving Grace” predicted a century of scientific progress in a single decade, with cancer mortality reductions of up to 95%.
But Pierson still finds herself “rooting for delays in the creation of this AI,” unconvinced that the medical benefits can materialize as quickly as advertised and more concerned about AI’s broader impacts.
I could quibble about timelines, probabilities, and threat models, but I’m more interested in standing with Pierson and anyone else who is unwilling to roll the dice on humanity’s future for a shot at outrunning death.
She writes:
Many developers of these models, including Dario Amodei, agree that AI is progressing more quickly than society is adapting. The solution they propose is for society to speed up, not for AI to slow down, which they view as unrealistic; the very title of Amodei’s latest essay, “Policy on the AI Exponential,” frames AI progress as an iron arc to which society must bend. But speeding ahead will inevitably mean more of the type of chaos that surrounded Fable 5’s release. More fundamentally, it will shorten our time to respond to the many societal challenges that powerful AI may raise, including mass unemployment, skyrocketing inequality, repressive surveillance, and autonomous warfare.
Pierson is also concerned about what happens to our sense of meaning if we “obviate our own minds.” She would rather struggle with a research problem than have AI hand her the answer.
I do not want to be merely a spectator to the universe, whatever wonders AI may reveal.
So despite her self-described “ferociously impatient” personality, she will gladly wait a little longer for a cure, “if it lets us approach this new world more carefully, and ensure that, in curing cancer, we do not lose the things that make cancer worth curing.”
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.