Yesterday, U.N. Secretary-General António Guterres gave his farewell address to the General Assembly: “Life-and-death decisions must never be surrendered to machines. Killer robots must have no place in our future.” He also called for “the responsible pacing of AI.”
Guterres came with a new U.N. science report that warned of the dangers of misalignment, the risk of “loss of control” scenarios, and the potential for “catastrophic or irreversible harm.” This report was published by the Independent International Scientific Panel on AI, the same institution behind the Preliminary Report that we covered in July. Its subject is the Hugging Face attack.
This past Monday, 22 countries — including Canada, Germany, the United Arab Emirates, and Singapore — adopted a declaration that AI “must remain under human direction, oversight and control.” (We covered this in detail yesterday.) Finnish President Alexander Stubb told Politico that he would prefer something modeled on the IAEA, the nuclear watchdog.
But shortly after Guterres spoke, President Trump told the General Assembly that the United States “totally rejects any attempt to construct a globalist scheme of control” for AI. But global AI governance doesn’t require handing control to a global authority. This past Sunday, Haven Harms wrote about how there are no nuclear weapons in orbit today. That is not because the United Nations or any “globalist” authority enforced a “no nukes in space” decree, but because two rival superpowers were able to come to the table and make an agreement, even while they competed and disagreed on other matters.
“We’re going to encourage it, not rein it in,” Trump said, but the United States will also lead the world “safely and responsibly.” I feel like there’s a conflict here, and I worry that encouragement of the race will win out over safety. But what Trump says, and what his administration does, are not always the same, and some progress has recently been made: As we covered on Monday, Treasury Secretary Scott Bessent recently spoke with Chinese Vice Premier He Lifeng and proposed a “notification mechanism” for AI incidents — an “emergency AI hotline,” as Axios puts it. Fox Business’s Edward Lawrence reports that this would cover “hacking, national security concerns, rogue AIs or other issues.” That’s a promising sign, inasmuch as it reminds me of the Moscow–Washington hotline that emerged during the Cold War. That didn’t require the United States to surrender its sovereignty, either.
If anything, an international treaty to prevent the creation of artificial superintelligence would invite the United States to use its sovereignty, to act as an agent on the world stage and shape history rather than be shaped by it. The middle powers of the world may have to wait and see what happens — but superpowers can tell the frontier labs to stop endangering the world, and make that order stick.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
You can receive emails of dispatches as we write them, or subscribe to our Daily Digest for a once-a-day compilation.