The risks of making sand think
Google DeepMind's Demis Hassabis pitches an AI regulatory regime
Exactly one month ago, Dean Ball, former White House AI adviser and current Head of Strategic Futures at OpenAI, wrote that the U.S. has an informal licensing regime for AI, albeit one without consistent rules or firm boundaries. This morning, Demis Hassabis published his proposal for a formal version.
The CEO of Google DeepMind proposes that a U.S. standards body for AI be created, similar to the Financial Industry Regulatory Authority (FINRA), which oversees Wall Street. In an interview with Axios, Hassabis said he would like to see this board be majority-independent and staffed with Turing Award winners and other experts.
This new body — and the benchmarks it develops — would then determine which models qualify as “frontier-class.” Stricter guidelines would apply to the “Frontier Labs” behind these models, for example, regarding security screening of personnel and safety budgets. The Frontier Labs would also be required to have all new models tested by the new body 30 days prior to release, with a focus on cyber, bio, and deception risks. Passing these tests would eventually become a prerequisite for releasing the model on the U.S. market. The system is designed to regulate frontier models from around the world without stifling startups or academic research.
Credit where credit is due: This is probably the boldest and most concrete proposal for a regulatory regime a frontier lab CEO has made to date. Hassabis has given thought to the fact that such a licensing regime must be as universal as possible and that the benchmarks must be designed in such a way that the frontier labs cannot specifically train their AI for them in advance. This is important so AIs don’t adopt behavior that maximizes training scores rather than what the engineers wanted those scores to represent, as we recently discussed.
Above all, however, he mentions quite casually and hidden in the middle of a paragraph that one task of the institution he proposes could be to coordinate a slowdown of the race, “if the seriousness of the situation demands.” This seems surprisingly vague given that it’s one of the most important parts of his proposal. Who would make this call? How would that happen? And is the situation not already serious enough?
Let’s take a look at how he describes the situation we currently find ourselves in:
At the moment, we are locked in an extremely intense, multilayered commercial and geopolitical race. While these competitive dynamics fuel rapid progress and accelerate the incredible upsides, advances on the frontier are outpacing our understanding of the technology. Nobody in the world knows for sure what is going to happen from here, and even the experts disagree.
He goes even further. Hassabis writes that while he believes the technical risks are solvable, this is only possible if industry and society give themselves the necessary time — and that we are currently failing to do so. It is the strongest and most compelling passage in his post, and it refutes his own proposal. After issuing this clear warning, predicting ten industrial revolutions at ten times the speed for the next decade, discussing recursively self-improving AIs, admitting that no one knows exactly what lies ahead… his proposal is to proceed with “cautious optimism,” a voluntary 30-day preview testing window and a new benchmark?
Furthermore, a race dynamic cannot be resolved unilaterally. Sure, on paper, this proposed U.S. body is supposed to have universal jurisdiction over all models. But would China’s Z.ai submit GLM 5.2 for 30-day preview tests just to gain access to the U.S. market? Would Beijing let them, even if they wanted to?
Hassabis describes the possibilities of AI in poetic terms. He speaks of “dawns” and “miracles,” the “foothills of singularity,” and about “making sand think.” He speaks of the risks in far more prosaic terms. But the brighter the light, the sharper the shadow. If we take seriously what he says about what’s at stake, then it seems necessary that we take the time to get this right. Hassabis has written a strong case for stopping and stapled it on a plan to race on. There is no such thing as racing cautiously. They taught sand to think and now it’s running through our fingers.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.