Dispatches from Alana
If we pause AI, how do we verify that others do too?
An article in TIME today discusses the idea of an AI pause modeled on Cold War nuclear de-escalation practices. Some AI companies agree (or at least say they do) that a pause would be good. But they also express concern that if they pause, others won’t.
I think the “if we don’t build it, someone with worse values will” narrative is deeply flawed. An AI does not absorb the values of whoever is standing next to it. These are black box systems with black box values. That said, I think the “Pausing doesn’t work unless everyone pauses” narrative is correct. Because it doesn’t matter who builds artificial superintelligence: if anyone does, everyone dies.
So the means to verify that all countries are indeed pausing is crucial, just as we needed verification measures like seismographs, satellites, and tamper-proof cameras to ensure compliance with nuclear arms agreements. The TIME article says the AI version of this technology doesn’t yet exist, but is being worked on by a group of startups. One of these startups, Lucid Computing, is building verification mechanisms using AI chips:
The idea is that a special piece of software could sit inside these trusted environments, where it could examine the AI and check whether it complies with a given rule. For example, it could confirm that a specific model is being run, or determine whether chips are being used in the training of a new model, which might be outlawed.
If tech like this is built in time, it could be game-changing. But the article also points out an added complication: verifying restrictions on AI is in some ways more difficult than with nuclear power because we don’t always know what to check for.
There are all kinds of different ways of measuring the capabilities and behaviors of AI systems. These measures are frequently subjective, and they become outdated rapidly ... until companies or countries get around a negotiating table, the startups trying to build verification technologies are pointing at uncertain targets.
While the article raises great points, I think it overstates the technical and policy challenges, at least a little. These challenges are a key research priority for MIRI’s Technical Governance Team, and they’ve published multiple proposals aimed at solving (or getting us closer to solving) exactly the kind of problems outlined in the article. They’ve done significant work on verification mechanisms, and the proposed solutions include methods we could implement with today’s tech. They’ve also drafted a sample treaty pointed towards the goal of banning the development of superintelligence globally, with proposals for how restrictions could be defined and compliance verified.
In short, while there’s no doubt we need more technical development and policy research into these areas, we shouldn’t wait to start pursuing an international pause. We can, and should, start discussing such a pause now. At the same time, we should start implementing the verification measures we already have, and developing the technology to make them stronger. Cold War nuclear verification was never perfect, but it was good enough for arms-control agreements to work.
Don’t go there
Talk radio host Hugh Hewitt’s opinion column in Fox News today can be summarized by the 90s band Sublime:
We’re only gonna die from our own arrogance. That’s why we might as well take our time.
Hewitt reminds readers of a 2011 piece by former Washington Post columnist Charles Krauthammer, “Are we alone in the universe?” Advanced civilizations are very likely to destroy themselves, the piece posits. Recounting a view cited in Krauthammer’s piece, that’s perhaps why there have been no signs of intelligent life elsewhere, despite math that would seem to favor its existence (Fermi Paradox, Drake Equation).
Krauthammer goes on to identify intelligence as perhaps “the most cursed faculty in the entire universe — an endowment not just ultimately fatal but, on the scale of cosmic time, nearly instantly so.”
All of this was written before AI took off, of course. But as Hewitt notes, his words surely apply to our current situation. When it comes to AI, Hewitt says, we’re basically in a car with the accelerator floored, unable to brake or even lift our foot off the gas. The destination? AI that exceeds human-level intelligence. And we need to slow down.
Krauthammer calls politics “the driver of history” and states it “will determine whether we live long enough to be heard one day.” Hewitt interprets this as meaning that it’s up to us to decide the future. I agree. If we don’t change course, we seem likely to be headed toward our own destruction, the end that Krauthammer hypothesizes is the destiny of most advanced civilizations. In the words of Hewitt:
It seems destined to end in the silent cosmos with the most recent contender to survive [us] lost in infinity of time and space. That’s not inevitable. Only extremely probable.
Our choice?
Get a grip on the wheel of “AI” or give yourselves over to a nightmare that doesn’t end well, and not just for us, but our children and grandchildren.
Speaking to Fox readers, he also anticipates that some might trust it will all work out regardless of what we do, believing that God has a plan, and we are mere pawns in it. To those readers, he states:
What does God expect of mere mortals staring at the tree of the knowledge of good and evil?
The answer: Don’t go there.
APA releases guidelines for patients using AI
The American Psychological Association is responding to the growing number of patients who are depending on AI chatbots for various aspects of their mental health.
I’ll share some notable findings from their 2026 survey of 12,000+ licensed psychologists below. (Note that this is a survey of mental health professionals, which means it excludes people who use chatbots for mental health but aren’t also in therapy.)
77% of psychologists say their patients use AI for support, whether that’s assistance in therapy, affirmations and behavioral reminders, or even self-diagnosis.
22% said their patients conversed with chatbots for friendship and 13% said they did so “for intimate relationships.”
Over a third of the psychologists surveyed said patients were developing chatbot dependency, a quarter said they were engaging in unhealthy conversations, and 15% reported patients developing distorted thinking or delusions.
I’m not particularly surprised by these numbers. Though this shouldn’t be taken as an endorsement of the practice, I’ve asked chatbots for medical advice myself, and found them to be surprisingly helpful. Sure, they can make mistakes. But so can licensed professionals. And when you can’t get a licensed professional, a chatbot often seems to provide better advice than a Google search.
But sometimes, a chatbot being “helpful” is exactly what can make relying on it harmful. Because developers don’t really know how to make something “helpful”; they can make it competent by training it on large swaths of data, but “helpful” is more subjective. So the quality is often measured, in training, by whether a user likes the response. And unsurprisingly, we tend to like answers that agree with us. Basically, instead of: “be helpful” the chatbot likely learns: “elicit a positive user response.”
That’s why I think the APA’s guidelines for patients using AI should be required reading for everyone, not just those using AI for mental health. The throughline: remember, chatbots are designed to keep you engaged and make you feel good. That drive will color their responses.
In therapy settings, a chatbot may give you what you want (ex. reassurance) but not necessarily what you need (ex. the ability to sit with uncertainty or fear). At worst, it can nudge you into psychosis or provide false medical information. In legal settings, AI might make up cases to help you make your point. In research settings, it might make up sources, or validate half-baked conclusions. Nobody wants it to do those things; it’s simply a result of our steering limitations.
Among my favorite reminders from the APA guidelines? “Feeling better does not always mean getting better.” Chatbots may make us feel great in the short-term. But this won’t always translate to long-term wellbeing, whether that’s mental, cognitive, or creative.
Dispatch from Joe
Money and politics
A tense New York primary comes to a head today, as underdog candidate Alex Bores seeks the Democratic nomination for NY-12 in Manhattan. Bores, who has cosponsored state AI regulation and whose campaign we covered previously, has a reputation in AI safety circles for taking the risks and benefits seriously. Several personal friends of mine have taken time to canvass for Bores, making POLITICO’s recent coverage of the race particularly salient to me.
The race has garnered mainstream coverage in part because of the massive spending by the Leading the Future super PAC in an apparent attempt to make an example of Bores. We’ve covered the dirty tactics of this group before; their opposition to Bores is unsurprising, given their rather explicit founding mandate “to support candidates aligned with the pro-AI agenda and ensure America leads the world in AI innovation and oppose those that do not.”
Quoted in POLITICO, OpenAI CEO Sam Altman and a PAC representative try to paint their spending as a counterweight to that of more safety-focused groups. But they have it exactly backwards; Leading the Future publicly targeted Bores last November, and the first pro-safety funding didn’t begin until this February (though it has since risen dramatically).
High super PAC spending is not unusual — POLITICO reports Bores’ opponent Lasher received $10M from one super PAC, for instance — but the amount of AI money in the NY race still stands out.
Overall pro-safety and anti-safety funding is fairly evenly split between Democrats and Republicans, a sign that AI regulation is still significantly bipartisan.
Despite this, Congress remains split on recent attempts to regulate AI. POLITICO covers the current mix of online child-safety bills. A Senate proposal backed by the current administration met with resistance in the House. The details can be difficult to parse, but it looks to me like this disagreement is due at least in part to yet another attempt to smuggle in sweeping federal preemption of state AI laws.
Whoever wins the NY-12 nomination and the subsequent general election, I hope for all our sakes that they and their colleagues push for sane regulations and a durable international halt to the AI race.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.