The new plan by the AI Futures Project, AI 2040, has earned a mention in the Washington Post. I expect we’ll see more coverage soon.
Yesterday, I heartily recommended reading their proposal for governing AI, Plan A. Today, I’ll share a shortened version of their story for how the future of AI might go well, a forecast which researcher Richard Ngo called “selectively optimistic.” Paraphrased, it runs like this:
The U.S. recognizes that the AI race is lethally dangerous, and reaches out to negotiate like we once did with the Soviets. China proves receptive because they’ve been worried about AI for a while.
Backed by the U.S. and China, an international agreement requires tracking the (highly bottlenecked) supply chain for advanced AI chips. A pause in training new frontier AI models is enforced by monitoring large datacenters.
(This step assumes that, in the next few years, we improve the technology that lets us verify that datacenters aren’t doing particular kinds of training. Without such tech, datacenters would have to be shut down entirely. The MIRI technical governance team has worked for years on this and related problems, so that governments have the right tools when they need them.)
Glad to see the U.S. and China slowing down, other countries join what comes to be called The Consortium. They work out a deal to proceed with AI development, but slowly, carefully, and with all research and AI algorithms open to public scrutiny. Frontier AI models themselves are publicly accessible, but their weights are not published online, lest they be used by bad actors to design bioweapons or restart the AI race.
New datacenters are built, but in places where they can be easily reached and shut down if the agreement falters (e.g., Chinese datacenters in Canada, American datacenters in Mongolia). AI developers have to make a robust “safety case” for their models (a dramatic improvement over today’s widespread carelessness).
By 2031, AI progress is going much slower than it could be, but the world still feels like a sci-fi story. AIs and robots are doing a large chunk of all work, regulated and limited to closely monitored regions to prevent it from completely dominating the economy. GDP rises massively, and taxes on machine labor fuel a “citizens’ dividend” that helps offset the loss of human jobs. Governments (advised by AI) invest in biosecurity and other precautions.
Other AIs, now as good as top experts, help human researchers figure out how to solve the fiendishly difficult problem of alignment, and make future AIs care about human things in the right ways. Several tense, increasingly weird years later, the world is ready to build superintelligence.
I have my concerns with Plan A, but perhaps my strongest disagreement is here. I don’t think we will end up with AIs who are smart enough to solve alignment, but too dumb to successfully rebel. We need a very different approach than the current grown-not-crafted machine learning paradigm.
I elided a lot of details here, in the name of brevity, and I renew my recommendation to go directly to the source for more. Despite my disagreements, I think AI 2040 represents a well-grounded vision of the future.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.