If they were somehow benevolent
A demand from Sanders, a manifesto from Zuckerberg, and an invitation from Texas
In this issue:
Bernie Sanders demands AI companies pause - His ability to be this direct and not be ignored is a sign of changing times
Zuckerberg shares shrewdly spun optimism in new manifesto - If everyone had a superintelligence...
Where the data centers will end up - The Permian Basin’s commercial landowners will warmly welcome the data center you keep out of your community
Dispatches from Mitch
Bernie Sanders demands AI companies pause
His ability to be this direct and not be ignored is a sign of changing times
In a widely reported cross-platform push today, Senator Bernie Sanders issued a blunt demand to the CEOs of OpenAI, Anthropic, and Meta:
Mr. Altman, Mr. Amodei and Mr. Zuckerberg: In the interest of humanity, stand by your words. Pause AI development. It is not too late to avoid disaster. Stop building machines that humans cannot control.
The demand came in the form of a letter, which the senator also read in a short video.
The letter opens with mention of the overtly rogue AI behaviors that have come to light over the past few weeks, as well as the AI-designed virus milestone. Sanders pairs these with the CEOs’ backing of employee demands to build a “pause button,” as he puts it — a reference to the Pacing the Frontier petition signed by more than 1,300 frontier AI company employees calling for the development of governance mechanisms needed to orchestrate an AI slowdown.
Responding to a 2025 OpenAI statement that it would “halt further development” if a “critical” threshold was ever reached, Sanders writes, “That moment is here.”
He’s not asking:
Let me be very clear: If you do not take appropriate action now, my colleagues and I in the U.S. Senate will.
To be fair, analysts aren’t putting much stock in that threat right now, given the current dynamics in Congress. But Sanders certainly isn’t the only policymaker who has been paying attention to AI news. Support for a pause feels like the kind of mood shift that could sweep across the Hill rather suddenly once a threshold is reached. New facts of concern are sure to come to light as the Hugging Face incident and other attacks are investigated.
Early in his ongoing push for a national moratorium on AI data centers, Sanders explicitly said that slowing AI development was “exactly the point.” Data centers were an easy target, being so unpopular even with those who aren’t worried about AI. The fact that Sanders can now call directly for a pause on frontier AI development without being ignored is itself a sign of changing times.
Zuckerberg shares shrewdly spun optimism in new manifesto
If everyone had a superintelligence...
I have to respect Mark Zuckerberg’s media savvy even as I loathe almost everything he says and does in his capacity as Meta’s CEO. Surfing the recent wave of support for open-weight models, he put Meta’s trailing AI program back on the media’s radar with the simultaneous release of an open model and a cheery 6,500-word manifesto about “personal superintelligence for everyone.”
The model, called Muse Glimmer, looks strong for its size on first glance, but that size is tiny. It’s not trying to compete at the frontier.
The manifesto is the bigger drop. Expanding on the thesis of his recent Wall Street Journal op-ed, Zuckerberg cleverly frames experts’ concerns as reasons to race faster and more freely.
Early in the doc, called “The Future is for Everyone,” Zuckerberg expresses surprise that “the discourse from many developing AI is so filled with doom,” saying he doesn’t understand “why anyone who believes that AI will eliminate most jobs and much of humanity’s relevance would rush to build that future.”
Did you catch it? The idea that AI can only lead to bad futures if the builder believes it can? It’s the same logic used in Meta’s recent advertisement — People are worried, but we choose to believe it will all work out, aren’t we brave? — marking the new essay as part of a broader branding exercise.
The manifesto is full of these shimmering rainbow oil slicks where optimism is passed off as reason for optimism.
Take this passage about the risks of AIs building smarter successors:
There are a few ways a balance of power could exist with recursive self-improvement. The simplest is that multiple labs could achieve RSI around the same time. If some or all of those superintelligences were somehow benevolent, then they could check and balance each other.
If they were “somehow benevolent?” If wishes were fishes we’d all swim in riches.
I think the very next thing he writes is worse, though:
That said, it is not clear that there is any way to expect benevolence or rely on superintelligence not directed by people for a positive future.
It’s bad because it looks like responsible admission that nobody knows how to control superintelligence or align it to something good, but all it’s actually saying is that humans need to stay in control of superintelligence — something his whole vision requires.
That vision doesn’t hold together. Whenever he’s talking about your free personal superintelligence, it’s a guardian angel that gives you business ideas, cures your cancer, and helps you bond with your family. When it’s China’s superintelligence, or the government’s, it’s out to stifle your growth and suppress your freedoms. Personal superintelligences are simultaneously aligned to their individual users and to keeping their users from using them to harm and oppress each other. These superintelligences are never mismatched in capabilities, never work in teams to overpower each other, and never pursue their own agendas.
How do we get these? By reaching “a state where billions of people are using and scrutinizing personal superintelligence agents.” No, really: He says that if we do this, “then we will have solved alignment to individuals’ interests.”
How could we have missed something so simple? How did we not know that the way to solve alignment is to get to a place where we’ve solved alignment?
We should treat Zuckerberg’s manifesto as the cognitive hazard it is. But there are a few concrete proposals in here worth mentioning:
One is that we should just decide that it’s ok for companies to distill each other’s models, in order to “protect the principle that you can learn from anything you can observe.” I guess that’s at least consistent with the idea that AI companies need not ask or compensate the authors and artists whose materials were used to train their models.
Another proposal is for the frontier AI labs to share “intermediate training checkpoints” of new models with the government for evaluation, rather than waiting until training is complete. I think this is actually pretty sane, though his reasoning is that even a 30-day review “may cede America’s lead and create worse outcomes.”
He additionally proposes that, in areas like biorisk, we focus on “limiting the physical production and distribution of harmful materials,” because the knowledge of how to synthesize these materials will stop being scarce. This is also a reasonable ask, though it’s part of his larger argument against trying to regulate the models democratizing the dangerous knowledge.
Of the many commitments Zuckerberg makes for his company in this essay, the most meaningful, if real, is his claim that:
Meta is implementing a governance structure that gives our independent board of directors the power to approve the safety criteria for releasing models and reviewing whether each model release adheres to the criteria.
I don’t expect this to help with my sleep quality. The last few years have taught us that neither companies nor their boards can be counted on to respect their own red lines.
Near the end, Zuckerberg writes that:
Developing superintelligence will be the most profound technological advance we will see in our lifetimes.
In keeping with the sophistry on display, I note that this claim can be true even if superintelligence kills us all.
Where the data centers will end up
The Permian Basin’s commercial landowners will warmly welcome the data center you keep out of your community
It brings me no joy when I have to tell people that local data center opposition, like a popped AI bubble, won’t do much to stop the race to superintelligence, and that we’ll need strong international agreements instead.
But it’s true. Just as the AI chips would flow to whatever post-bubble companies could make best use of them, data center projects in exile will land wherever they are received. It is inconvenient to the tech giants to lose their preferred sites, but there’s no shortage of federal or commercially owned land in the middle of nowhere.
A Wall Street Journal piece points to the Permian Basin of West Texas as one such promised land. Roughly the size of Nebraska, but with fewer than 500,000 residents, the region is home to companies that bought great “land empires” of scrubland in order to lease out the mineral rights beneath. Now, these holders think they can make a fortune from the surface, too, by leasing it to data centers and selling them “everything from water to construction materials.”
In an indication that these plans are expected to materialize, the stocks of these land companies are up as much as 56% this year.
Data center builders like CoreWeave have signed deals, and so have energy companies like Chevron that want to supply them. Matchmaking is expected to be easy in the region due to a shortage of pipelines for bringing its abundant natural gas reserves to market. Data centers in the basin mean the gas can be profitably burned for electricity near where it is produced.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.