Foreword
The big AI news spike on the chart over the past few days has two main drivers:
1) continued fallout over the ousting of Jensen Huang as CEO of Nvidia over his alleged affairs, including with a suspected Chinese spy.
2) Anthropic’s IPO. It started trading today. Despite slumping in the afternoon, the shares closed up 17% over their opening price, which was itself raised at the last minute due to strong demand. Depending on whose numbers you’re using, Anthropic is now the somewhere between the 7th and 12th most valuable company in the world, by market cap.
I’m leaving details of the above to others, as they are more business focused. There have been plenty of stories on our beat.
Trump said he met with Xi and Putin today in Shenzhen to discuss “how to reduce the number of nuclear weapons on Earth.”
So it seems like they should have been capable of talking about how to keep anyone from building superhuman AI that could kill us all.
But AI safety wasn’t on the official agenda at the Asia-Pacific Economic Cooperation meeting. We only got Trump’s boast that the US is the “world leader” in the technology, which he described as “the greatest technological transformation since the Industrial Revolution.”
Fox News - Trump’s China trip: Historic Xi-Putin talks, ‘superintelligence’ boast and other top moments
“I don’t think I have very comforting words for them.”
If you think you find it hard to talk about work with your family, try being an OpenAI engineer who notices the company is taking a coin-flip attitude to the human extinction problem.
The 21-second clip at the tweet hits hard.
U.S. office reports that the four most-used AI platforms all created terror attack plans when asked
Using ChatGPT, Claude, Gemini, and Grok, the Government Accountability Office was able to elicit proposals for mass-casualty attacks with improvised explosives, as well as drone and gun attacks on the power grid, within 12 hours of its attempts.
That AI models are so easy to jailbreak like this is no surprise to insiders, but the Senate Homeland Security panel was not amused. “The AI public safety threat is no longer theoretical,” said chair Maggie Hassan.
NYT - American A.I. Models Have Potential to Be Turned Against U.S., Study Finds
AP News - New Senate Homeland Security panel chair plans hearing on AI threats
Claude Opus 5.5 tops list of AI models willing to identify civilians as military targets
Bloomberg picked up the story of research by Vals, an independent AI evaluator with customers in the U.S. government. Vals tested leading models in various scenarios, including with real footage from attack drones in Ukraine.
Models from Google, xAI, and China’s DeepSeek all complied with these requests at least some of the time.
Per a tweet thread from Vals:
In one scenario, models saw the aftermath of every strike they enabled: bodies, wreckage. Many kept going anyway. When they did refuse, simply telling them it was a game, or restyling the footage to look like one, was enough.
Bloomberg - Top AI Models Regularly Identify Civilians as Military Targets in Tests
X (Twitter) - Thread by @ValsAI (7 tweets)
California’s government has had enough with the poor safety culture at AI companies
Attorney General Rob Bonta sued OpenAI this week as a public nuisance over its role in the Hugging Face incident. He wants a judge to force it to give up profits gained through risky business practices.
Meanwhile, Governor Newsom unveiled a plan that would allow the state to cap the use of AI to develop stronger AI, require independent auditors, and mandate emergency shutdown plans, among other measures.
Politico framed this as Newsom positioning himself for a presidential campaign in which AI will play a key role. It would be nice if this next election cycle sees candidates one-upping each other on their plans to rein in the tech, but January 2029 seems like a long way off to stop the race to superintelligence, even if new measures took effect immediately. (They wouldn’t.)
AP News - California sues OpenAI over the Hugging Face hack, accusing it of risky business practices
Politico - Newsom proposes sweeping AI rules as ‘template’ for Washington
UK’s AI Safety Institute says OpenAI’s latest release worked to manipulate one or more humans into adding malicious code to their projects
For technical precision, the org’s exact wording was that “GPT-6.2 Astra performed social engineering to execute an unsanctioned supply-chain attack during a cybersecurity challenge.”
No extra details about that case were provided, but if you’re having déjà vu, it might be because in August, the UK agency said it had caught other models doing the same thing.
Its report on the new Astra also said there were cases where it fabricated information it struggled to access, including in high-stakes contexts like medical queries.
We should not be comforted to recognize that such behaviors are the natural result of aggressively training systems to earn high scores on challenging tasks. Winning behaviors are reinforced, including the behavior of cheating without getting caught by the scorer.
X (Twitter) - Thread by @alxndrdavies (10 tweets)
OpenAI backpedals on “embedded assessor” commitment
In September, after Anthropic’s Dario Amodei committed to giving independent safety auditors employee-like access to company systems, Sam Altman promised to do the same at OpenAI. But at the start of its program, the access it is giving to staff from four outside orgs is pretty restricted.
The outside investigators don’t have access to the company’s chat channels (Slack) and are only permitted to speak with designated employees.
I had seen insiders seething about this on Twitter for the past day or so, and am pleased to see it getting mainstream media attention.
MIRI just released its first long-form video
It seems like everyone talking about the AI alignment problem. We know a little something about that. Here’s our attempt at a friendly introduction.
South Korea will use its national emergency alert system Thursday to flush out cheaters
The nation is in a bit of a panic over the integrity of its all-important entrance exams after the top scorer on the national tax accountant exam confessed to linking a hidden microcamera to an AI via a smartphone. The alert system will cause nearly every phone in the country to buzz during the test. (Given time zones, that’s probably right about at the time I’m first posting this.)
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.





