“It’s what we’ve all been saying,” wrote one Anthropic researcher about the alarm raised by Jacob Coxon in his viral resignation announcement.
The New York Times’s Mike Isaac and Kate Conger gathered several such statements for an article yesterday about the worried chatter inside AI companies. That one came from a company chat. But employees across labs are reportedly taking their concerns off the record as well, in “encrypted chats and private dinners,” with intent to organize awareness-raising efforts. One existing group, the Coalition of Concerned AI Staff, is described as a project to create “stronger safeguards” and hold companies to their safety commitments.
Publicly and privately, employees are debating the impact they can have by loudly quitting like Coxon vs. applying leverage at their companies. They are almost universally choosing the latter. Why? After Coxon’s resignation, workers reportedly “urged top executives at their companies to pause development of the technology so they would have more time to work on it in a safe way.” But I didn’t hear them threatening to quit.
I think it’s because industry staff have been weighing these issues for a long time. Those deeply bothered at the idea of personally accelerating the AI race had quit already or never taken a job at an AI company in the first place.
The same is true of the AI executives themselves, which is important to remember when people ask why AI CEOs would be actively building something they think could kill us. There were always going to be people determined to build superintelligence. These were going to be people who believed it was possible. To make superintelligence your life’s work, you kind of have to take the idea seriously. Taking it seriously means recognizing how dangerous it could be. We should be much more surprised by founders working toward superintelligence who don’t think it’s dangerous.
Mark Zuckerberg is arguably the exception that proves the rule. The Meta CEO uses the word “superintelligence” a lot and talks as though it’s only as dangerous as your beliefs about it. But he was also late to the race and seems to see it as a mere consumer product. His visions of a future with superintelligence are substantially more mundane than those of his peers — bordering on Jetsons territory — suggesting he doesn’t actually believe in the kind of superintelligence his own engineers are working on.
It’s crazy that people can try to build a machine god without so much as a permit. But we’re in this situation only because lawmakers and the public spent the last twenty years dismissing the idea of superintelligence in our lifetime. The kind of people who stuck with their grand project through all the mockery mostly aren’t going to be the type to quit now that the laughing has stopped.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.