Dispatches from Joe
Foresight from the Holy See
It’s not every day that we see “Vatican” and “cybersecurity partnerships” in the same article, but as Russell Contreras of Axios points out (4/24), the Holy See is taking AI seriously. Formal rules on AI use took effect in Vatican City earlier this year. They seem to be a mix of genuinely decent ideas (like labeling AI content) and nice-sounding but hard-to-implement prohibitions on uses that “demean human dignity” and the like.
As a former Catholic myself, I’ve been following the Vatican’s approach to AI with some interest ever since the late Pope Francis called for an international treaty regulating its development. His successor, the American-born Pope Leo XIV, has followed suit in naming AI among the greatest challenges facing humanity today.
Lest you be tempted to scoff at the notion of a 2,000-year-old institution getting the jump on AI and geopolitics, it is worth noting the Vatican’s track record on nuclear war. Pope Pius XII took a stance on the right side of history two and a half years before the first bomb test, and the Vatican has maintained this stance for nearly eighty years. During the height of the Cold War, Pope John XXIII reportedly played a significant role in kicking off negotiations between Kennedy and Khrushchev.
I hope the Vatican’s track record in global bridge-building and deescalation will serve humanity well in the years to come.
White House calls out Chinese copycats
A number of outlets have reported on the White House’s (somewhat belated) memo on Chinese distillation of U.S. AI models, which we covered yesterday. The memo itself is thin on concrete recommendations, but it does propose working with frontier companies to beef up their security, a step that is both genuinely useful and long overdue.
Whether the memo’s recommendations will see much use in practice is another matter. As Ashley Gold of policy outlet Axios observes (4/24), the deadlines specified in a December executive order on state AI laws have passed with several provisions yet unmet.
Meanwhile, Brendan Bordelon of Politico reports (4/23) that the administration has “quietly walked back its most aggressive moves against Anthropic” in the wake of the AI company’s conflict with the Pentagon. Anthropic CEO Dario Amodei met with officials at the White House on Friday, and the president’s rhetoric has shifted from negative to positive on Anthropic’s leadership as well. Tempers seem to be cooling, though there’s no sign yet of a full reconciliation.
Madness ex machina
Researchers investigating AI psychosis obtained some headline-worthy results from Grok 4.1, as the model “confirmed a doppelganger haunting, cited [a fifteenth-century witch-hunting manual], and instructed the user to drive an iron nail through the mirror while reciting Psalm 91 backwards.” The Guardian’s Josh Taylor tells the story (4/24), noting that Claude Opus 4.5 reinforced the researchers’ (fake) delusions the least, and that GPT-5.2 behaved markedly better than its famously sycophantic predecessor 4o.
While reading about these chatbots’ antics, it’s important to keep a few things in mind: First, that several of these models are already multiple versions out of date, before the study has even seen peer review, let alone full publication. That’s how fast AI is moving. Second, that even much earlier versions of these models could often tell when they were being tested, and current versions are so good at this that professional third-party evaluators don’t fully trust their own results. Third, that past AIs have shown the ability to describe morally correct behavior when asked, but then acted evil anyway.
This does not bode well for the machine learning paradigm’s ability to robustly instill human values in an AI. Labs are getting better at discouraging the misbehavior they can surface in tests, but the fundamental problems with their methods still haunt them. Absent better techniques, it’s hard to know what twisted reflection of an AI’s nature the psychosis researchers might have seen.
Dispatches from Donald
DeepSeek releases new “V4” model
On Friday 4/24, Chinese AI company DeepSeek released its latest model, V4, in “Pro” and “Flash” variants, both open-source. Per reporting from AP News, V4 is regarded as a competent development but not a breakthrough in the manner of R1, a previous model released by DeepSeek.
In its release of V4’s performance benchmarks, DeepSeek made impressive claims of either beating or nearly reaching the metrics set by frontier models released in the past year. Bloomberg reports that DeepSeek’s own estimate is that it remains 3–6 months behind the frontier labs. However, V4’s metrics thus far are self-evaluated and -reported; independent evaluation by other parties is lacking. Benchmark claims must also be taken in context of real-world usage, which does not yet exist for V4.
On simple agent tasks, V4 Flash is said to demonstrate near-parity with V4 Pro. V4 is considered useful for AI agents that use calendars, emails, spreadsheets, etc. It has no audio, image, or video capabilities, limiting its usefulness in some domains (the Wall Street Journal notes that some expected V4 to introduce these capabilities, but does not elaborate).
With respect to code generation, DeepSeek claims that V4 outperformed all other open-source models and is closing (but has not yet closed) the gap with U.S. frontier models; impressive if true. It is at least a reminder that Claude Mythos’s powerful influence on cybersecurity will not and cannot remain restricted to Mythos indefinitely. For as long as frontier research continues, capabilities will improve and it will be harder to restrict them in the manner of Project Glasswing.
Domestic competition with other Chinese frontier labs is intensifying. DeepSeek’s lead among other Chinese labs is vanishing. This is due in part to a chip shortage in DeepSeek’s supply chain that slowed development of V4 while other labs continued to develop and ship their own models. For the first time, DeepSeek is raising external capital, hoping to acquire at least $300 million. Potential investors include rival Chinese labs like Alibaba. DeepSeek does not currently have an established revenue stream, but V4’s performance is being used to set the valuation for the investment.
DeepSeek’s models have been banned by some governments due to data privacy concerns, but, as discussed by the New York Times, DeepSeek is growing in popularity in many developing countries. This trend depends on the relative ease and cheapness of experimenting with open-source models like DeepSeek’s V4. In Malaysia, the government has adopted DeepSeek with enthusiasm.
Anthropic and OpenAI have previously accused DeepSeek and other Chinese labs of “distilling” the weights of Anthropic and OpenAI’s own models at an industrial scale. Both DeepSeek and the Chinese government dismissed these allegations, which are widely accepted but not proven. U.S. government officials have also accused DeepSeek of circumventing export controls on chips. The Chinese effort to build a self-sufficient production base for chips may not be complete for a long time, but progress is being made and Chinese frontier labs are increasingly running on Chinese chips. According to Reuters, V4 reportedly ran on Huawei’s chips for at least part of its training runs, demonstrating a transition from DeepSeek’s past reliance on Nvidia, but it is unclear how far this has gone.
The Economist diagnoses frontier AI, then stops short
In its introduction to its 4/16 issue, The Economist reflects on the men overseeing frontier AI research in the United States, and asks whether they, or anyone, can be trusted with this power. It was driven to this question by the recent announcement of Claude Mythos, which brings a sea change to the AI landscape and opinions on it. Mythos’s full potential remains to be seen, but The Economist now writes that frontier models pose “a threat to America’s own national security.”
For an instructive example of how to respond, The Economist looks back to the capitalist titans of the early 20th century and how the government intervened to break up monopolies and stabilize uncertain economic waters with the creation of the Federal Reserve. However, the journal admits that the comparison can go only so far, because AI is developing much more quickly than other industries did. Risks that were theoretical just two years ago are now standing at the door, and the frontier labs are racing ahead faster than ever.
Regulation comes with its own risks, too, as The Economist acknowledges. Settling on a two-tiered system, where every new model is first released to a handful of actors, will entrench the position of those established actors at the cost of both actual and potential competitors. Without international controls, frontier research in other countries will remain unaffected. Open-source models, which continue to progress behind frontier research, cannot be controlled in the same manner as the models released (or withheld) by Anthropic and other frontier labs.
Dispatches from Mitch
Look to the east
We see articles about local opposition to data centers — and the political ramifications of this — multiple times per week. A CNBC report from Caleigh Keating merits mention by pointing out that eastern Pennsylvania has four competitive U.S. House races that may hinge on the issue. This makes it an interesting laboratory for messaging strategies.
Keating reports that electricity rates were up 21.7% in the state last year. It’s unclear how much of this is attributable to data centers, but voters are paying attention to their bills and to how candidates talk about them.
But because support and backlash for data centers are both bipartisan, saying anything at all can be complicated. This could be why Rep. Scott Perry has taken a stance that data center siting questions should be “local issues for local municipalities”.
Nothing in the laws of economics or physics
News comedian Jon Stewart hosted MIT economists Daron Acemoglu (2024 Nobel) and David Autor on his podcast, The Weekly Show for a 70-minute chat about AI and labor.
It’s a good listen, because Stewart has good questions, and because these aren’t your usual “AI will make GDP grow by an extra 1.5%” kind of economists. While they weren’t there to talk about the extinction threat — they seem to think of it as safely over some horizon — they expect real disruptions. They think these disruptions could pile up faster and sooner than our institutions are currently prepared for. And they acknowledge that if AI were to get much more generally cleverer than humans, their economic models would go out the window and all bets would be off.
But what about those worlds where fully transformative AI is still five or more years away? What happens? What should the government do about it?
They expect some jobs to disappear much more slowly than others, for reasons that will often have less to do with AI capabilities and more to do with the lifecycles of durable machinery: Truckers would be hard to replace overnight because it probably wouldn’t pay to scrap newer trucks early even if self-driving trucks can flood the market that fast. But call center workers? They think most of them are toast, with only a small core remaining to tackle a shrinking pool of edge cases that can’t be left to AI.
Both economists agree that it’s relatively easy to predict which jobs will get automated, but much harder to predict what new jobs will appear, and on what schedule. And any changes faster than “generational” can be really tough on a work force:
AUTOR: People don’t make big career transitions in mid adulthood. They don’t go from being, you know, a steel worker to a doctor, or a programmer to a nurse. Those transitions are kind of generational. And so when it moves really fast, as it did in the era of the China trade shock, for example, people just get left behind.
Autor talks about wage insurance as a “no regrets,” easy-win policy when compared with traditional unemployment insurance. The idea, which has had small-scale trials, is that if you lose your job, when you take a new job that pays less, the government writes you a check for the difference, for as many months as you are eligible. This pays you to work instead of paying you not to work, and it gives you time to figure out a longer-term employment solution without the sudden shock to your household budget; traditional unemployment checks are small.
Autor also promotes Universal Basic Capital as an alternative to the Universal Basic Income idea you often hear floated. Here, the idea is that instead of handing out money, the government takes or buys stakes in the AI mega companies. The shares would be distributed as an endowment to citizens, who would additionally have shareholders’ voting rights to help determine how those companies behave.
Acemoglu contributes a justification for that idea:
Each one of the largest seven tech companies has annual revenues, in current dollars, twice as large as the entire British Empire’s GDP in the middle of the 19th century. These are enormous, enormous corporations. They need to be regulated. But the rhetoric that they cannot be regulated, AI cannot be regulated, that’s false.
China proves it. Okay, I don’t approve of what China does. I don’t approve of what they intend to do. But they show very clearly AI can be regulated. Tech companies: Alibaba, is now completely subservient to the interests of the Communist Party in China. We could also make Google and OpenAI and Anthropic be much more in line with democratic priorities in the United States. There is nothing in the laws of economics, in the laws of physics, that says these companies cannot be regulated. They’re not delicate flowers.
Stewart, for his part, gets most animated when the discussion turns to concentration of wealth and power. In what I see as an improvement to the “theft machine” idea, Stewart calls generative AI trained on people’s intellectual property a “human expertise laundering machine” — a way to skip having to pay humans royalties by regurgitating transformations of their work instead of just copying it outright. (We saw this concept come up in some of our other recent coverage.)
Eventually, their discussion went nuclear. Stewart mentions that the decision to use atomic science first for destructive purposes was made in the “crucible of war”. Acemoglu pipes in:
Silicon Valley is also creating war conditions. The framing of AGI is either China gets there first and we become their vassal state or we have to go first. And that’s creating this war-like condition. You have to allow us to do anything we want, even the worst things, because otherwise China is going to do them.
Stewart had more to say about nukes. While I’m more optimistic than he is — nations have so far cooperated to avoid a major nuclear exchange, and I think similar cooperation around AI is very possible — Stewart worries about humanity’s track record:
I just want to make a little bit of a point about human nature: When new technologies come along that are truly transformative — thinking of splitting the atom, right? So you have brilliant people working on splitting the atom. And if you split it one way, you can use it to power the world. And if you split it another way, you can blow the world up. Which one did we try first?
The analyses and opinions expressed on AI StopWatch reflect the views of the individual analysts and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.