Politico attempted to map out some of the main factions within AI safety, a term that has come to mean many different things.
I could quibble about the labels and groupings, but I’d rather point out that even with the (questionable) breakdown Politico created, there’s a surprising amount of common ground. Four of the seven categories include people who are very concerned about the risk of extinction from AI. Nobody likes the term “AI safety.” And most people think we should make sure we can keep AI under human stewardship. The main exception seems to be the so-called “let it rip” crowd, which was an odd choice for inclusion in an AI safety field guide, since they view safety as antagonistic to innovation.
Granted, the individuals featured almost certainly disagree about how to keep AI under human stewardship. Some think guardrails and safety checks will be enough. Others think we need to stop development until we have a better understanding of how to reliably get AI systems to do what we want. (I’ve written about why available guardrails and safety checks aren’t currently up to the task, and my colleague Robert wrote about the limits of tests just yesterday.)
But there’s still widespread agreement on a common goal, which is certainly not something to take for granted. That gives me some hope.
One place I will quibble with the article is its description of MIRI president Nate Soares as a proponent of “completely shutting down AI development.” His frequently stated position is that “the only thing that needs to stop here is this race to create artificial superintelligence.” In other words, we should stop developing more and more powerful systems until we understand how to reliably steer them; this could mean pausing for decades or more. During such a pause, Nate says, companies could still pursue more narrow AI for certain domains: things like cancer research, drug discovery, and self-driving cars.
While Politico’s summary of Soares’s stance wasn’t quite right, the quote they included from him was spot on:
I’ve never liked the term ‘AI safety.’ If you’re in a car careening towards the edge of a cliff, it’s not the time to bring up ‘car safety.’ Car safety is about seat belts; it’s about airbags; it’s about crumple zones; and none of those will save you if the car goes soaring off the cliff. AI companies are racing to create artificial superintelligence, and the AIs they’re creating aren’t carefully programmed to follow instructions; they’re a mess of trained numbers that even the creators don’t understand. What we need is not a little extra ‘AI safety.’ What we need is to stop this reckless race.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.