Top forecasting team recommends AI slowdown
Five plans for an AI future, and why the AI race must end
Intelligence, some argue, involves two distinct but related tasks: predicting what might happen, and steering (the future) towards what you want to happen.
Last year, a team of writers, researchers, and forecasters, with quite a lot of intelligence between them, made a rigorous and thoughtful effort to predict the future of AI, titled AI 2027. The educated guesses that they made have held up well so far, and the members of the AI Futures Project have continued to regularly update their model as new knowledge comes to light.
Today, with a suite of policy recommendations they term AI 2040, also known as Plan A, this team of experts tries a more direct form of steering. They have charted a path they hope the world can follow, a path they predict leads to far better outcomes than the alternatives. In their own words:
AI companies are racing to build AIs that are smarter than humans in every way. In AI 2027, we predicted that this would result in either extinction or irreversible concentration of power.
Plan A is our positive vision for what should happen instead.
I have some disagreements with Plan A, but they can wait. For today, I wholeheartedly recommend reading the whole thing.
They don’t pull punches in the intro:
The industry has convinced itself that controlling superintelligent AI can be figured out on the fly, and thus has no remotely adequate plan. We think this situation is terrible and could easily get us all killed. [...]
We hope critics will judge us against the existing state-of-the-art for plans to navigate the AI transition (if they can find any) and not against some hazy but pleasant fantasy where no one has to make any hard choices yet everything will probably be fine.
(I would nominate MIRI’s draft international agreement and the tentative path in its Appendix as one example of the state of the art to beat. It has a notably different focus than AI 2040, but there are many points of agreement.)
I must admit, I love their report. In addition to sensible proposals delivered in a clear and engaging style, it has all the bells and whistles: clean footnotes, interactive charts and breakout boxes, and a choose-your-own-adventure-style layout for five of the many paths the world might soon take.
What are those paths? The first three are pointedly terrible in various ways: let companies race full-speed to build superintelligence; burn a months-long AI lead for scraps of safety; or nationalize the companies and fight tooth-and-nail to slow China. Then there are the two plans that seem to have a decent shot: negotiate a verifiable slowdown with China and proceed with caution (the authors’ Plan A) or take it one step further and shut the whole thing down (Plan S).
The authors seem genuinely torn between the last two, but they come down on the side of Plan A. If I understand them right, they have two main reasons: first, they think there’s a reasonable chance near-superhuman AIs can help align future AIs to human values (instead of to the AIs’ own values, or instead of failing at this hard task), and second, they worry that even a well-implemented Plan S could break down into a worse version of other plans, putting us right back where we started.
I’ll cover my disagreements with this conclusion in a future post, but for now, I commend the authors for spelling out their reasoning so clearly. In a world where policymakers take heed, and seriously debate just how intensely to slow down AI, I am much more optimistic about our odds of surviving as a species.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.