I am not generally an emotional man, but the news I read this morning moved me almost to tears. Seeing in the Washington Post that U.S. Senator Bernie Sanders (I-VT) and Representative Greg Casar (D-TX) had announced the Ban Artificial Superintelligence Act, I felt a mixture of hope and trepidation. What sort of policies go with the provocative name? Would they be sensible, or slop?
After I read the details, it was hope that won out.
The legislation is forthcoming, so we don’t yet know the full text, but the summary does not mince words. The bill would outlaw attempts to develop or deploy superintelligence, defined as:
An artificial intelligence system that exhibits or can easily be modified to exhibit capabilities that match or exceed human cognitive performance and capabilities across a broad range of domains or tasks, [or] AI systems that have sufficient capabilities to plan and execute the disempowerment of humanity, including by overthrowing or undermining the U.S. government.
The bill would also:
Halt “advanced AI development” until a federal regulatory body develops clear standards, enforced by corporate shutdown and jail time for violators;
Establish “a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence” and assign an advisory board of AI experts; and
Set U.S. international policy on a path towards international agreements that make this ban global.
I am impressed. I have some concerns — we’ll get there — but this is the first piece of proposed AI legislation I’ve seen that might drastically improve humanity’s chance of survival.
The proposals are mostly common sense: We should not build something smarter than us that we lack the means to understand or control, and we should stop private companies who attempt this. AI technology is far too important and dangerous not to have a dedicated federal agency, and a panel of knowledgeable experts would help non-technical authorities set better policy. Of course, a unilateral U.S. halt isn’t enough when other countries continue, so international coordination is urgently needed.
Some open concerns temper my enthusiasm. First of all, I’m aware this is likely something of a messaging bill, unlikely to pass in the current session of Congress. But I’m going to take its message seriously, and look at what we know from the summary so far.
The definition of superintelligence looks reasonable as written, but “advanced AI development” is much less so, and may be overbroad. I still respect the decision to err on the side of caution while no serious regulation exists.
Speaking of regulation, I’m unsure whether the regulatory body and the “cabinet-level agency” are intended to be separate entities. My guess is that they are, which means that a fully fleshed-out bill ought to include explicit language and a timeline for creating both.
It makes sense that the proposed federal agency would “Monitor frontier AI systems at all stages of the lifecycle” and “Supervise the removal of dangerous capabilities,” but I’m not sure what is meant by the third responsibility, “Supervise the destruction of artificial superintelligence.” The main way humanity defeats a superintelligence is by not building it in the first place. But I can imagine some scenarios where a nascent, dangerous AI, not yet fully superhuman, exists alongside a government-ready kill switch, and it does make sense that this agency would be charged with the shutdown.
I would also make the agency — or perhaps the regulatory body — responsible for research into technology that can verify international agreements. A key part of such deals is making sure everyone keeps them, like the International Atomic Energy Agency helps do for nuclear nonproliferation agreements.
The proposed international policy is thin on details. Understandable for a one-page summary, but a real policy will need more flesh. Starting off with a unilateral U.S. halt, without negotiating a deal with China first, may not be the best method; but it still seems better than the current hands-off approach, and it might inspire China to treat superintelligence with similar gravity.
In 2024, my colleagues in the MIRI technical governance team drafted a more gradual plan for securing international coordination, published in an appendix to a proposed international agreement. (Disclosure: I also had a hand in editing both.) It’s only an example, though, and there are many different paths the world might take to a global halt.
Ultimately, even if the proposed bill dies in committee, it still represents a massive breakthrough in sane AI policy, and a huge leap relative to what was being considered a couple years ago. We are finally beginning to see what it looks like for the AI problem to be treated with respect, at least from the policy side.
I look forward to seeing the full text.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.