In a New York Times op-ed today, Jen Easterly, director of the U.S. Cybersecurity and Infrastructure Security Agency (CISA) from 2021 to 2025, warns that “We have a dangerous habit of taking warning signs seriously only after a catastrophe has taught us what they meant.”
As an example, she gives the “failure of imagination” cited by the 9/11 commission as the reason the CIA failed to connect the “blinking red” dots before the 2001 terror attack. She finds similar precedent in the 1986 space shuttle Challenger disaster and the 2011 Fukushima nuclear meltdown.
Little imagination is required in the case of AI, but people seem determined to let the inevitable happen before acting:
Today many warning signs are emerging from the world’s leading A.I. labs, with companies racing to build systems of immense power, with little meaningful regulation. The familiar response in these situations is to wait for an A.I. system to cause consequential harm — an autonomous cyberattack that significantly disrupts access to power or clean water or a model that helps a terrorist build a biological weapon — and only then hold hearings, appoint a commission, impose new requirements and ask why we did not act sooner.
She proposes a weekly council, convened by the federal government, where technical leaders from leading AI labs would meet with top government intelligence and security officials to “examine incidents, near misses and newly discovered capabilities across proprietary and open-weight models, including intelligence about capabilities emerging from China and other foreign developers.”
There is precedent for such a council. As director of CISA, she had overseen the formation of a Joint Cyber Defense Collaborative that was like this, but for cyber concerns. But it’s important that, unlike that group, an AI risks council not be “just another discussion group.” It must be wired into White House authority and decision-making.
Her concluding words are potent:
None of this requires believing catastrophe is inevitable or even likely. It requires accepting that if the worst outcomes are preventable, the moment to prevent them will necessarily come before the evidence is complete, before the costs of delay become obvious.
We should not confuse uncertainty with safety. We should not wait for the warnings to become wreckage. And we should not need another commission report to tell us, after the fact, that the systems were blinking red.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.