Last night, AI researcher Jacob Coxon announced on X that he is resigning from Anthropic. One of the reasons Coxon gave for his resignation was that neither Anthropic nor OpenAI are acting responsibly. Instead, he says, they are gambling with all of our lives by racing toward artificial superintelligence.
In his thread, Coxon writes many things with which I wholeheartedly agree, and they should sound quite familiar to regular readers of AI StopWatch:
I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.
His arguments — while very good — are not new. They are essentially exactly what the AI safety community has been repeating for years, some for decades now. Jacob Coxon is also not the first frontier lab employee to resign in order to warn of the significant risks — Nobel Prize winner Geoffrey Hinton (formerly of Google), Daniel Kokotajlo (formerly of OpenAI) and Richard Ngo (formerly of OpenAI and Google DeepMind) have done the same. Over 1,300 employees of frontier AI companies have signed the Pacing the Frontier statement. In the statement’s personal comment section, many researchers have been similarly forthright.
This time, though, Coxon’s message seems to have come at exactly the right moment and is currently gaining significant momentum. As I write this, his post on X has over 100 million views and more than 500,000 likes. I can’t recall a warning about the extinction risk posed by superintelligence ever receiving more attention and support on X.
Meanwhile, several U.S. senators, representatives, and even some British members of Parliament have taken Coxon’s post as an opportunity to join the chorus of warnings and call for more regulation. The governor of Illinois, JB Pritzker, writes:
It’s time to sound the alarm—louder—on reining in artificial intelligence. It’s becoming increasingly clear that AI poses a threat to humanity, so I’m calling for immediate action from the industry and Washington.
Even flagship media outlets — not only in the English-speaking world but around the globe — have picked up on the warnings and are reporting with unusual seriousness on the danger we face.
I’m still a bit surprised by how strong the reaction suddenly is, but I’ll take it. Gladly!
Of course, the reactions aren’t unanimously positive. In addition to some of the usual bad takes, there were also, bizarrely enough, a few voices on X who initially doubted that Jacob Coxon even exists or that he accurately reflects the views inside the frontier labs.
Evan Hubinger, the current Alignment Lead at Anthropic (and former MIRI researcher), came to his defense:
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think the probability is >10% within the next decade. I believe Anthropic is doing its best, but we don’t yet have a plan to solve alignment for superintelligence and aren’t clearly on track to do so.
So we shouldn’t assume that the increased attention will automatically make it easier to convince people. But we should do what we can to build on this momentum.
If you’ve been waiting for an opportune time to call or write to your senator, representative, or local member of parliament, to demand that they take action against the danger posed by uncontrolled superintelligence, this is it.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.