Over the past week, my colleague Joe has twiceencouraged our readers and listeners to engage with AI 2040: Plan A, the new scenario from the AI Futures Project.
Now it’s my turn.
I won’t retread the ground Joe has covered, but I will note that one of the passages Joe quoted (possibly my favorite) has already proven prophetic in the worst way. The passage reads:
We hope critics will judge us against the existing state-of-the-art for plans to navigate the AI transition (if they can find any) and not against some hazy but pleasant fantasy where no one has to make any hard choices yet everything will probably be fine.
Alas, the chatter I’ve seen about this plan on X (Twitter) has been overwhelmingly of the hole-poking variety, without giving alternative answers to the hard choices ahead.
Hard choices do have to be made! That’s true even in the futures where we’re getting pretty lucky, where it’s proving suspiciously easy to keep powerful AIs from stealing our planet to pursue the strange preferences their training will unintentionally but unavoidably cause them to develop.
Plan A is already easy mode. It takes place within one of those suspiciously lucky futures where AIs can be trusted to do our AI alignment homework for us, designing more powerful and more trustworthy AIs that humans themselves could not. Only in those futures will it be enough to slow AI for a mere decade or so, as the Plan proposes. The group’s Plan S — which they say they are sympathetic to and think might be better than Plan A — is the “simpler” plan that tries to prevent superintelligence for as long as it takes humans to know how to get it right on the first and only try, which could be a few decades or more.
Both plans depict the balancing act that awaits us if we slam on the brakes but don’t turn the car around. We avoid plunging straight off the cliff, but still creep towards it, picking up speed because the AIs that will already have been created by the time a pause takes hold will be powerful enough to transform the world — economically, militarily, and socially, if not in the literal extinction-causing sense that we’d get with superintelligence.
It’s a balancing act because these transformations will be destabilizing. On one side of the beam: war, chaos, anarchy. On the other side: exploitation, tyranny, oppression. Maintaining the balance will require hard choices about things our institutions weren’t built to handle. What happens when total surveillance is cheap and easy? When nuclear deterrence is undermined? When every terrorist can design a bioweapon? When individual companies are more powerful than nation-states? When a huge fraction of the population is unemployable? Critics of Plan A are calling it naive, but assuming business as usual will work out fine is the most naive plan of all.
As others have pointed out, the rate of economic and technological progress depicted in Plan A is still blazingly fast — faster, indeed, than most AI boosters expect business as usual to go, because boosters tend not to grasp AI’s full implications. So theoretically, those who want to accelerate AI should be just as happy with Plan A as those who want to slow it down.
Anyone complaining that Plan A is a luddite stagnation fantasy needs to read the Public Perspective section at the end of it. This is introduced as a “shorter, more accessible story we wrote to convey what Plan A would feel like to a typical American,” and it’s wilder than almost all of the sci-fi you’ve ever been exposed to. It’s not until 2028 that you finally lose your job and start seeing robots walking around in numbers, doing real work, but by 2034 you’re living in a luxury high-rise in a city that didn’t exist five years earlier, and by 2040 you’re pondering whether you want to remain in your biological body and what you might want to do with the ten-billionth slice of the universe you stand to receive.
And that’s just the beginning. Read the Epilogue and Space Governance Proposals for the truly wild stuff.
You may not like Plan A, Plan S, or any plan you’ve seen so far. Cool! I don’t wholeheartedly love any plan I’ve seen yet, either. But if you’re not down with something like Plan A or Plan S as a starting point, what’s your alternative? If you don’t have one, who do you trust to come up with a better one for you, and how will you empower them to enact it? The choices are coming, and they must be made.
Maybe you’re looking at all of this and wishing humanity didn’t have to face any of these choices in your time. Okay! How will you not just stop the car, but turn it around? I think it’s possible. You might even be able to talk me into supporting it.
But you’re going to need a plan.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.