Dispatches from Mitch
The traitors and terrorists opposing new data centers
I’m troubled by recent news and opinion pieces that remind me of the McCarthy era, when people with views the U.S. government didn’t like were flagged as communists, and communists were aggressively persecuted.
Except today it’s not communists; it’s traitors or terrorists, and one of the views the government doesn’t like is opposition to AI.
This morning’s Washington Post ran an op-ed by retired Air Force general David A. Deptula with the headline, “Blocking the construction of data centers is a national security risk.” Americans, in his description, wouldn’t do such things, because they “wouldn’t want China to win the AI race or to see U.S. forces outpaced in a fight over the Taiwan Strait.” The unspoken implication is that data center opponents are traitors.
That would line up with U.S. Interior Secretary Doug Burgum’s remarks on Fox Business’s Mornings with Maria Tuesday. There, he implied that residents opposing data centers in their communities are sabotaging the U.S. at the behest of China:
[8:42] You know, China is our arch rival here. China, with streamlined or no permitting at all, can just blow through all the regs and build what they need to build. In the U.S. we’re finding opposition, but one of the things — Kevin O’Leary reported on it last week, more people should be — but there’s a concentrated information propaganda war that’s geotargeted; any place that’s trying to build data centers is getting bombarded with foreign-directed propaganda to try to block these from being built. So this is again just another attack on the U.S. and our ability to be competitive.
Law enforcement also seems to be looking at data center opposition as a threat to the homeland. WIRED’s Daniel Boguslaw reported Tuesday on more than 1,000 pages of documents released in response to a Freedom of Information Act request. These reveal that the FBI, DHS, and fusion centers — offices serving as go-betweens for federal, state, and local law enforcement — are monitoring a new domestic threat category: “anti-tech violent extremism.”
This follows Trump’s National Security Presidential Memo 7, which directs the Justice Department to target movements with “anti-American,” “anti-Christian,” and “anti-capitalism” beliefs.
Documents showed fusion centers monitoring school board and town budget meetings where discontent over data centers is expressed. Northern Virginia’s fusion center catalogues anti-data center protest activities such as “photography,” “observation/surveillance,” “expressed/implied threat.”
Boguslaw also found that a for-profit intelligence contractor, SITE, flagged a video from the progressive nonprofit More Perfect Union. The video, discussing a data center’s impact on Georgia residents, contained no call to violence, but is “now circulating across the country as a potential threat vector.”
Lest this all sound too much like a conspiracy theory, I’m not claiming any coordinated effort behind the threads in this dispatch. When I say the “government” doesn’t like AI opposition, I don’t even mean all of government, only certain factions within the executive branch. But these are the factions calling most of the shots right now, and their words matter.
And for the record: I don’t think opposing local data center construction is a strategy that will do much to slow the race to artificial superintelligence. But I respect anyone doing what they can to protect themselves and their families, within the confines of the law and common decency.
“Are you, or have you ever been, opposed to the race for artificial superintelligence?” is not the sort of question that should be used to brand someone a traitor. There is no prize for America waiting at the finish line, only ruin for us all.
North Korea’s “AI” missile, and the difficulty of outrunning one’s shadow
We’ve previously reported on the way individuals, companies, and even whole countries overclaim their degree of AI use in an effort to look ahead of the curve.
I’m adding North Korea to my list of examples.
Breitbart reported that North Korea’s government-controlled news agency boasted of an “AI terminal guidance function” in recently tested cruise missiles. Terminal guidance is the final stage of flight where the weapon, having arrived in the neighborhood, must identify its exact target and steer to it as precisely as possible. Systems for this that match camera or radar images to programmed target profiles were developed as far back as the 1980s.
I would bet good money that North Korea’s announcement is a rebranding of existing missiles using these more traditional systems, or even just plain old GPS. That’s not because I don’t think North Korea is using AI — we know its hackers are. It’s because, for a poor nation, I doubt current AIs improve missile guidance enough to justify the added complexity and on-board power consumption.
If you hear someone using North Korea as yet another reason to race faster on AI, don’t be surprised; just know that it’s a silly argument: Even more so than China, North Korea’s AIs are only as good as we give them. The U.S. raises the floor on the capabilities of every adversary every time we allow one of our companies to release a better open-weights model that anyone can use and modify. We do the same any time we allow a company to make a frontier model accessible on a platform with insufficient guards against distillation — a process by which foreign rivals can crudely clone its capabilities.
Yes, China is now releasing open-weights models on par with the best American open-weights models. But China’s overall capabilities are still piggybacking on what can be distilled from American products, aided by the theft of American research and the rampant smuggling of American AI chips. Under these conditions, our rivals will never be more than a few steps behind no matter how fast we race. We might just as well try to outrun our own shadow.
America’s strongest AI safety bill
As reported in WIRED, the Illinois legislature just passed what’s being called the toughest AI safety bill in the U.S., with Governor JB Pritzker saying he’ll sign it into law.
In the words of Scott Wisor, a policy director at a non-profit supporting the bill, SB 315 requires the state to “appoint an independent auditor to check whether the AI labs in fact adhere to their safety commitments.”
If you’re dismayed that the toughest AI law in the land merely compares the companies’ behavior to their stated commitments, join the club! The law can assess penalties of up to $3 million per violation, but given that these companies have market caps pushing $1 trillion and could just change their commitments, I don’t expect SB 315 to much improve their behavior.
That’s probably why OpenAI is endorsing the bill. This would be consistent with its chief lobbyist’s new “reverse federalism” strategy of creating a de facto federal standard through state laws that favor industry. Anthropic also endorsed the bill.
But for a trade group that includes Google, Apple, Amazon, and Andreessen Horowitz, even this is too much. They opposed the bill.
To be fair, the bill does require a few other things companies may find annoying, like incident reporting, transparency requirements, and whistleblower protections. So kudos to Illinois: This is progress.
Dispatch from Alana
Replacing humanity on purpose
According to a report from Vox today, the billionaires actually want to kill us.
Of course…not all humans will make it through the transformation [when we “take AI off the leash” and task it with figuring out how to merge with humans]; only a select group of people will transition to the next evolutionary stage.
The man seated beside me, a researcher from one of the major AI companies, was even more radical. Forget merger — it’s okay if humans don’t survive at all, he said.
These are the conversations Sigal Samuel witnessed when he went for cocktails with a group of AI successionists after a symposium they held in September at the New York Academy of Sciences.
To be clear, these ideas are not new – Elon Musk (who has his own AI utopia vision) cited successionist talk from Google co-founder Larry Page as the reason they stopped being friends in 2015. But it seems like successionism is gaining popularity and influence.
What’s an AI successionist? As summarized by Samuel:
A subculture … who think that artificial intelligence is our rightful heir — the next step in cosmic evolution. Since they believe AIs could become our moral superiors, they argue it’s actually wrong to try to keep the machines down, or even to align them with human values, as most AI companies aim to do. Instead, we should usher in artificial intelligence as a successor to humanity and hand over the world to it. Even if that means we go extinct.
Who would go to a successionist symposium? The event, dedicated to the idea of creating a worthy successor for humanity, included attendees from Anthropic, Google DeepMind, and xAI. It was also attended by “people from think tanks that directly shape the US government’s AI policy.”
I sincerely hope at least some of those attendees were there for intel rather than solidarity. But Samuel implies the latter, saying the AI successionist view is “becoming highly influential” and naming venture capitalist billionaire Marc Andreessen and Y Combinator CEO Garry Tan as some of the people with power, money, and influence who support their mission. The successionists have also “successfully cozied up” to the Trump admin.
I think most of this speaks for itself. But I want to make a few points:
1. It’s not just unchecked incompetence that could lead to our demise; in some cases, our replacement is the deliberate goal.
We’ve talked previously about how today’s AI systems are black boxes that can’t be reliably steered, yet are getting ever more capable. How even relatively limited AI systems display concerning behaviors, and how testing comes up short. This is more than enough for me to advocate for halting frontier AI development until we know more. But the successionist view takes things to a completely different level – some of the people building and funding AI work actually want it to replace humanity. And by “replace”, they don’t mean jobs or skills. They mean our species.
2. It should be fairly obvious that’s not okay.
Samuel spends much of his article diving into the philosophy behind successionism, arguing we need to counter it with something better, and exposing some of its holes:
What do we wish, ultimately, to become?
This is a moral question, even a spiritual one, and it demands a spiritual response. The AI successionists are offering one. For anyone who finds it repulsive, the challenge is to offer a countervailing positive vision.
(and later)
But if there is a better vision for our technological future than the one offered by AI successionism, what is it?
To this, I say:
There are billionaires intentionally – and literally – working towards our demise. We don’t need to do any philosophizing at all to realize that’s wrong, and to shut it down.
In other words, the burden of proof that there’s a better way forward than replacing humanity does not rest with the people who don’t think we should deliberately replace humanity.
More charitably, I will add:
It’s great to ask questions about what humanity should become and how society can improve. It’s also great to explore philosophical traditions that can help answer those questions. But if we wait for consensus on those topics before banding together to halt these mad billionaires, we’ll almost certainly be dead. We can’t philosophize when we’re dead.
So no, we don’t need to “create a new humanism before the AI successionists win.” We just need to make sure they don’t win. And that’s likely a lot easier than getting people to agree on the markings of a new humanism.
3. The successionists are making a big assumption about what a post-human world would look like.
I feel like most people don’t really need to hear this. But just in case anyone is compelled by the successionist vision of AI “spread[ing] that flame [of consciousness] far further than we mere humans can, generating experiences of bliss and forms of moral value that we could never even dream of”, let me remind you that AI does not have to be conscious to succeed humans, and likely won’t be. Whatever utopian world you might envision the AIs creating in the absence of humans is more likely to be a hollow, barren and joyless place devoted only to its own propagation.
Dispatch from Beck
Simulated Biology
People used to think protein folding might be an impossibly hard problem; then AlphaFold cracked it. A full explanation of protein folding is, in some sense, a full account of biochemistry, but suffice it to say that it governs the structure of the machinery inside cells. As such, it is important in medical fields like drug discovery. For their AlphaFold work, Demis Hassabis and John Jumper shared in the 2024 Nobel Prize in chemistry (along with David Baker for computational protein design).
Axios reports on BioHub, a Mark Zuckerberg and Priscilla Chan-funded institute, which released what they call a “world model of protein biology” that extends that work. BioHub’s release includes a protein folding model, a protein language model, and “ESM Atlas,” a map of 6.8 billion proteins and 1.1 billion predicted protein structures. This is a significant advancement in the number of proteins that science at least somewhat understands.
And, if it holds up to scrutiny, most importantly, the release allows users to design protein interfaces with sufficient fidelity that they can then “take them into the laboratory and they function as predicted.” This isn’t yet taking the lab out of the loop, but it’s moving it in that direction.
This route of simulated biology represents one important way future models can learn about the world. Folks who think that superintelligence will be slow-growing say that it will be bottlenecked by the need to run slow experiments in real life. But continually improving simulations will keep moving the ratio away from lab work and towards computation. Such simulated training is already at the heart of many advancements in robotics, like those of Boston Dynamics.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.