I tried to keep StopWatch out of this. One of our founding principles is that we will try not to fuel or instigate infighting among those working to prevent our extinction from AI. And this publication is largely aimed at people normal enough to have never heard of Effective Altruism (EA). But with the term reaching mainstream headlines, the time has come to talk about it. I apologize in advance for the offense I am likely to cause.
Ineffective selfishness?
I first considered writing this post last week, when an official Twitter account belonging to the U.S. Department of War posted a promotional image for an “AI-FIRST WAR DEPARTMENT.” The text of the post reads:
Americanism, not effective altruism. The United States will continue to be AI DOMINANT! [US flag emoji]
That wasn’t the first time I’d seen executive branch officials pushing Effective Altruism (EA) as a woke, Luddite boogeyman, but previous occasions had all seemed intended for an insider audience that would know what EA was. With this tweet, my heart sank in recognition that the administration was starting to push anti-EA messages to the general public.
I found the post super cringe, because I doubted even 1% of Americans have heard of EA. I imagined readers thinking, “I knew Trump officials weren’t big fans of altruism, but why are they rejecting effectiveness? If you’re going to do altruism, shouldn’t you at least try to be competent?”
EA 101
Effective Altruism is a fuzzy cluster of people who self-identify as interested in trying to combine moral imperatives with rational thinking to do more good in the world. EA first crystallized as a named concept around 2009. The classic example of EA in practice is the distribution of anti-mosquito bed nets in malaria-prone regions, which research showed to be a very cost-effective way to save lives. EA’s earliest and simplest incarnations are sometimes described as “spreadsheet altruism,” and EAs themselves as well-meaning nerds.
EAs recognized that coordination, money, and political influence are all very helpful for being effective, so they formed some strong interpersonal networks and well-funded organizations near the center of the cluster. Much of that funding came from EAs who intentionally chose lucrative careers or were early to cryptocurrency. (These are nerds, remember.)
But effectiveness is only half the story. The altruism part is more complicated, because to maximize the good you have to first decide what counts as good. Most EAs would probably say they’re trying to do some combination of “maximize flourishing” and “minimize suffering,” but they split over targets and strategy. Do animals and other creatures suffer? How much should we weigh this, given that they might be less conscious but more numerous? How much should we weigh the lives of humans who don’t exist but might exist in the future, depending on our choices? How far should one pursue complicated plans that would do a lot of good at the end, but which risk corruption and loss of purpose along the way?
With that last question in particular, you might see how the arc of Effective Altruism can bend towards supervillainy for those who overestimate the strength of their moral fiber. Some of the best people I know or write about are EAs, but so are some of the worst. The movement is seen to have peaked around 2020-21, because in 2022 it was badly tarnished by the fraud and bankruptcy scandal of cryptocurrency exchange FTX, founded by major EA donor Sam Bankman-Fried. Many EAs stopped calling themselves EAs after that.
EA and AI
What does this all have to do with AI and the White House?
Well, being nerds looking for more leverage to improve the world, many EAs were unusually early to recognize that AI was going to play a huge role in our future — maybe a dominant role.
For some EAs, the immense good AI might enable made rushing to superintelligence a moral imperative. Every day without it brings roughly 165,000 deaths from aging and disease, a much larger amount of potentially preventable suffering, and a slice of the cosmos made permanently inaccessible by the expansion of space itself.
For other EAs, the immense harm AI could do made AI safety the most important cause — not just for the 8 billion people on Earth, but for the trillions of beings who might never exist if superintelligence wipes us out.
Many EAs were intellectually complex enough to hold both views at once. Some of these were early investors or engineers at OpenAI before leaving to form Anthropic over safety concerns. Dario Amodei, Anthropic’s CEO, was one of them. These people may or may not actually believe what they say about how it would be better if we could slow the race down. But they definitely believe our chance of surviving superintelligence will be worse if someone else builds it.
All this is to say that EAs are simultaneously some of the people racing hardest for superintelligence and some of the people now trying hardest to slow the race down — or at least talking about it.
EAs are not, as a group, trying to stop AI any more than musicians, as a group, are trying to play jazz. Sure, a lot of musicians love jazz and think the world would be better for embracing it. But even the aficionados are mostly playing more profitable genres instead — sometimes in the name of funding their eventual, someday pivot to jazz.
The Trump administration dislikes Dario Amodei for his principled stances against autonomous weapons, domestic mass surveillance, and — most recently — his calls for industry safety measures that might eventually, someday, slow AI progress a little. Metaphorically speaking, they can tell he likes jazz.
Those trying to prevent our extinction from AI dislike Amodei because he’s still pushing the frontier despite how vocal he is about the risks. Metaphorically speaking, they’re still waiting to hear him actually play some jazz.
EA as boogeyman
Blaming a shadowy cabal for inconvenient developments is part of the standard political playbook, whether it’s a “vast right-wing conspiracy” or the Dominion voting machine conspiracy theory. (Fun fact: “vast right-wing conspiracy” was brought to us in 1995 by Chris Lehane, then a political operative working for the Clintons, now OpenAI’s chief lobbyist.)
When Jacob Coxon’s resignation triggered a sudden swell of public concern about the AI extinction problem, the Trump administration could have chosen to pivot to addressing said problem. It instead chose to stick with a full-speed-ahead agenda. This made it necessary to undermine the extinction concerns, which it did by signal boosting conspiracies already being promoted by industry accelerationists allied with the administration. Some of these theories revolve around AI safety groups run or funded by Effective Altruists. So today, the White House would have you believe that extinction concerns are lies spread by those woke, Luddite EAs.
EAs are mediocre in the boogeyman role. They’re hardly MS-13 or Tren de Aragua — groups which don’t look very good even on their best days. Factions within EA have even welcomed the attention, expecting that rational people looking into them would find much to like.
As cabals go, EAs aren’t very shadowy, either. But like other historically popular scapegoats, EAs are well-networked and well-resourced. If you’re into corkboard and red yarn, you could spend many hours in your shed drawing lines between AI company staff building AI, third-party researchers evaluating said companies, and AI safety orgs opposing them — they are pretty transparent about these connections. You could knit a sweater from the strands running through EA, because EAs got in on the ground floor. That’s what happens when you take AI seriously a decade before people start paying attention.
But even now, after the rogue swarms and the Coxon awakening, the typical, median EA probably isn’t trying to stop AI development. They’re probably not “woke” by any reasonable definition. And they’re definitely not a Luddite. If they’re involved with AI at all, they are probably trying to find a path forward that lets development continue with more testing and guardrails, telling themselves that we can creep closer to the edge without going over, and that everyone will of course pause when the time is right. Eventually. Someday.
And all that jazz.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
Great write-up, thank you.