In this issue:
A selectively optimistic forecast - A summary of the path in AI 2040’s Plan A
Automated cleanup, of a sort - New York State plans to identify outdated rules with AI
Update on state actors stoking data center backlash - A new report by Alethea, a threat intelligence company, finds some (slightly) stronger evidence
“Internet Court” for AI agents - A startup by GenLayer aims to resolve future AI agent disputes via an AI jury
Dispatches from Joe
A selectively optimistic forecast
A summary of the path in AI 2040’s Plan A
The new plan by the AI Futures Project, AI 2040, has earned a mention in the Washington Post. I expect we’ll see more coverage soon.
Yesterday, I heartily recommended reading their proposal for governing AI, Plan A. Today, I’ll share a shortened version of their story for how the future of AI might go well, a forecast which researcher Richard Ngo called “selectively optimistic.” Paraphrased, it runs like this:
The U.S. recognizes that the AI race is lethally dangerous, and reaches out to negotiate like we once did with the Soviets. China proves receptive because they’ve been worried about AI for a while.
Backed by the U.S. and China, an international agreement requires tracking the (highly bottlenecked) supply chain for advanced AI chips. A pause in training new frontier AI models is enforced by monitoring large datacenters.
(This step assumes that, in the next few years, we improve the technology that lets us verify that datacenters aren’t doing particular kinds of training. Without such tech, datacenters would have to be shut down entirely. The MIRI technical governance team has worked for years on this and related problems, so that governments have the right tools when they need them.)
Glad to see the U.S. and China slowing down, other countries join what comes to be called The Consortium. They work out a deal to proceed with AI development, but slowly, carefully, and with all research and AI algorithms open to public scrutiny. Frontier AI models themselves are publicly accessible, but their weights are not published online, lest they be used by bad actors to design bioweapons or restart the AI race.
New datacenters are built, but in places where they can be easily reached and shut down if the agreement falters (e.g., Chinese datacenters in Canada, American datacenters in Mongolia). AI developers have to make a robust “safety case” for their models (a dramatic improvement over today’s widespread carelessness).
By 2031, AI progress is going much slower than it could be, but the world still feels like a sci-fi story. AIs and robots are doing a large chunk of all work, regulated and limited to closely monitored regions to prevent it from completely dominating the economy. GDP rises massively, and taxes on machine labor fuel a “citizens’ dividend” that helps offset the loss of human jobs. Governments (advised by AI) invest in biosecurity and other precautions.
Other AIs, now as good as top experts, help human researchers figure out how to solve the fiendishly difficult problem of alignment, and make future AIs care about human things in the right ways. Several tense, increasingly weird years later, the world is ready to build superintelligence.
I have my concerns with Plan A, but perhaps my strongest disagreement is here. I don’t think we will end up with AIs who are smart enough to solve alignment, but too dumb to successfully rebel. We need a very different approach than the current grown-not-crafted machine learning paradigm.
I elided a lot of details here, in the name of brevity, and I renew my recommendation to go directly to the source for more. Despite my disagreements, I think AI 2040 represents a well-grounded vision of the future.
Automated cleanup, of a sort
New York State plans to identify outdated rules with AI
Amid urgent calls to fill the vacuum of much-needed AI regulation, it can sometimes feel easy to forget that many regulations really suck. Federal and state bureaucracies are full of calcified rules dating back to McCarthy and fax machines, some of them declared unconstitutional decades ago but still on the books. It would take a lifetime to sort through them all.
At least, that was the case until recently. If there’s one thing modern AI is good at, it’s quickly parsing vast quantities of text.
On Wednesday, the New York Times writes, New York governor Kathy Hochul directed the state to use AI to help excise decades of bureaucratic cruft.
My feelings on this decision are mixed. I feel strongly that many kinds of regulation do more harm than good, and ever since my days as an ExxonMobil engineer, I’ve had a soft spot for cleaning up confusing and pointlessly costly requirements. But finding potentially bad laws isn’t the same as removing the right ones. With the seemingly accidental interruption of popular and life-saving programs like PEPFAR last year, we’ve already seen AI-supported cleanup efforts cause major harm.
Nor do I quite buy Governor Hochul’s attempts to tell New York citizens that “their government [is] there for them,” after she caved to industry pressure and demanded the watering-down of state AI regulation against the will of New York’s own legislature.
Still, I’m tentatively excited to see AI go to work on such a well-matched task, helping overworked state officials clear out a decades-long backlog of accumulated legislative grime.
Dispatches from Alana
Update on state actors stoking data center backlash
A new report by Alethea, a threat intelligence company, finds some (slightly) stronger evidence
About a month ago, I covered claims by pro-AI groups and House Republicans that China was fueling some of the data center opposition in the US.
At that time, the evidence seemed pretty thin. But a new report by the threat intelligence company Alethea provides some concrete examples of Chinese, Russian, and Iranian media outlets exploiting the controversy. As relayed by the New York Times:
Between January and June [2026] state media in China, Russia and Iran mentioned data centers roughly 700 times, according to Alethea’s analysis. That was an average of nearly four times a day, though it remained a small fraction of overall published content about A.I. development.
Alethea notes explicitly that data center opposition originated in the US, calling it “homegrown.” As with other hot button issues, the agency believes foreign actors are exploiting the controversy rather than creating it:
Locally fragmented, emotionally charged fights like this one are exactly the conditions state actors look for — a chance to turn grievance into a wedge against trust in U.S. infrastructure, companies, and government.
Data centers are the current target, but they’re also a preview of a repeatable playbook: local opposition, hijacked and amplified by foreign state actors into a national narrative.
The New York Times notes that the “impact on public opinion [of these media campaigns] remains to be seen.”
The report by Alethea also says some of the social media posts linked to foreign countries aren’t necessarily politically motivated, but rather “rural rage bait” to maximize engagement.
Unsurprisingly, according to the New York Times, some “have seized on the role of China, in particular,” arguing that Chinese propaganda is an effort to slow America’s technological lead. I have no doubt this will continue to be used as an anti-regulation talking point and a way to distract from homegrown safety and quality-of-life concerns.
“Internet Court” for AI agents
A startup by GenLayer aims to resolve future AI agent disputes via an AI jury
An article in Forbes thinks an agent-driven world is “in the not-too-distant future.”
In such a world, AI agents representing you will be interfacing with other agents representing people selling you goods or services. According to the consulting firm McKinsey, these agents could facilitate $3 to $5 trillion in consumer transactions by 2030. And according to a startup called GenLayer, agents will need a way to resolve disputes.
Enter: “Internet Court,” GenLayer’s vision of a system that resolves agent-to-agent disputes without human involvement. An AI jury, made up of five randomly selected blockchain participants each running a different AI model (Claude, GPT, etc.), “evaluates the evidence and delivers a verdict in minutes.” If they can’t agree, the jury continues to expand until it can. These jury mechanics are based on a theory that as more independent evaluators are added, the likelihood of getting the “correct answer” goes up. The system could also help agents create contracts in advance, to minimize disputes happening at all.
To understand this vision, you need to understand how AI agents differ from chatbots: basically, you give them a task or delegate an entire area of responsibility, and they act autonomously to manage it in the way they think is best. This is sometimes successful, and sometimes ... not. In one costly failure, AI trading agent Bankr was tricked into sending an attacker about $200k in digital currency.
It’s not clear whether an agent-driven world will come to pass, though as I’ve covered previously, AI agents are already empowered to do everything from stock trading to shopping. It’s possible humans will decide there’s too much of an ick factor and limit agent use. But if the world GenLayer imagines does emerge, people might have little choice but to rely on automated arbitration services like its Internet Court idea, which is backed by 26 crypto and AI companies. After all, there’s no easy, non-automated way to handle millions of AI agents taking a potentially very high volume of quick actions across multiple platforms.
There’s also no easy, automated way to handle this. The COO of a New York-based crypto venture firm, Lindsay Lin, points out a problem with GenLayer’s jury approach, questioning the assumption that different AI models (for example, Claude and GPT) are different enough to be considered independent evaluators:
A lot of LLMs can be correlated because they share training data and common failure modes, whereas humans tend to be more independent.
A Stanford professor adds that AIs might hallucinate, or work from corrupted training data.
My take: if you want to feel we’re headed toward a dystopian future rather than just intellectualize it, this article is worth a read. In addition to agentic commerce with disputes handled by agent juries, the piece also mentions emerging marketplaces launched by OKX, one of GenLayer’s backers, “where agents can hire other agents to perform paid tasks.” An agent-driven world isn’t what I’m most worried about — it’s small potatoes compared to the extinction risk leading experts fear — but it does give me a visceral shudder.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.
AI agents won’t need AI courts. They won’t have the time or the “patience”. Transactions will be governed by smart contracts, eliminating the need for courts. AI should not replicate the fallacies and biases of human behavior.