Rules of engagement
Mythos expansion, "chipflation", search changes, autonomous weapons, and more
Dispatches from Mitch
Europe is finally getting Mythos
At long last, Anthropic is expanding its Project Glasswing to include trusted companies and government agencies in the European Union. The project gives cyberdefenders early access to the company’s dangerously cyber-capable Claude Mythos model. More than 150 organizations across 15 countries are included in the program expansion.
As we’ve previously reported, the Trump administration was understood to be the main blocker to European expansion. So it may not be coincidental that the President signed an AI executive order yesterday. The order asks AI companies to provide the U.S. government with early access for frontier models, but insists that this is purely voluntary and that the head start would not exceed 30 days (vs. 90 days in the previous draft). For reference, my archives indicate that the U.S. government has likely had access to Mythos since even before its initial reveal in early April, so probably at least 60 days ago.
In its announcement of Claude Opus 4.8 last Thursday, Anthropic said it has been working on stronger cyber safeguards for its products and expects “to be able to bring Mythos-class models to all our customers in the coming weeks.”
The things money can’t buy (at least not right away)
Two stories from the past 24 hours drive home the fact that AI companies are scaling up their compute as aggressively as they can, and that this is affecting you as a consumer.
In the Wall Street Journal, Katherine Blunt reports that more than 60% of data center capacity planned for 2027 hasn’t even started construction yet. That’s not for lack of AI demand, nor for lack of funding. It’s mostly due to permitting fights, electrical power constraints, and a shortage of computer components.
The hardware shortage isn’t from some external shock (though the Strait of Hormuz closure isn’t helping), but from unprecedented demand. It takes years to set up a new chip fabrication facility. AI companies are trying to outraise each other to outspend each other to outbuild each other. But what they’re actually doing is outbidding each other for the same scarce resources. Blunt writes:
It is a seeming paradox: If hyperscalers can’t break ground on many of the projects they have already announced, what difference can hundreds of billions of dollars more make—however eager Wall Street may be to supply it?
In a bidding war against the AI giants, consumers lose. As Reuters reports, Morgan Stanley has coined the word “chipflation” to describe what happens when memory chips cost six times what they did a year ago.
From the Morgan Stanley report:
What began as an AI infrastructure bottleneck is now spreading into hardware margins, device affordability, cloud costs, inflation and policy.
Most obviously, devices that use high-end computer memory cost more, when manufacturers obtain chips to make them at all. If you’re paying more for a phone or a computer, or waiting on a next-generation gaming console, or graphics card, or Valve’s Steam Machine and new virtual reality headset, you can put much or all of the blame on the AI race.
Less obviously, the chip price has chip manufacturers reconfiguring their facilities to make more chips for AI instead of other kinds of chips. This causes shortages to ripple into less cutting-edge products.
This slop isn’t for you
In her Bloomberg column today, Parmy Olson gives one of the sharpest takes I’ve seen about what it means for Google Search to push AI summaries over its traditional blue links.
The change doesn’t just steal traffic from the sites being summarized, it incentivizes new kinds of Search Engine Optimization — attempts to boost a site’s search rank by gaming Google’s algorithm. Traditionally, this is done through methods like creating or paying a bunch of sites to link to the target site as a signal of influence.
If instead of fooling a ranking algorithm you want to fool a large language model, the strategy changes. The clearest path to making the AI think that your site or products are better for users is to flood the internet with chatter and testimonials to that effect — especially on Reddit, a site known for its outsized influence in LLM training.
From now on, when you encounter a too-effusive, too-detailed product review, you shouldn’t just wonder if it was written by an AI, but whether it was written for an AI.
Dispatches from Alana
UK publishers can opt-out of Google’s AI summaries: will it help?
AI summaries — answers that appear at the top of Google search results — are lowering traffic to other websites. Users can answer their queries right away, without having to visit the sites the summary is based on.
In the UK, in an effort to level the playing field, web content publishers are now able to prevent their content from being used in these summaries. A regulatory watchdog, The Competition and Markets Authority, is responsible for the change.
Maybe I’m missing something, but I don’t think opting out of AI summaries will do much to restore lost traffic. Users are unlikely to take the time to scroll past the summary, even if it becomes less comprehensive than it already is.
Like refusing to use chatbots, I see opt-outs from site owners as a principled stance that seems unlikely to create real change without near-universal adoption. In the near future, I think we’ll need a much more direct approach to regulating what AI companies — and companies using AI tools — can and can’t do.
(To be fair, the Competition and Markets Authority also includes a directive for Google to properly cite publisher content, which — though quite a small step — seems unequivocally positive.)
You can’t program in morality... or anything, for that matter
A piece in the Guardian today asks: Can autonomous AI-powered killer drones take morality onboard?
The article quotes from a range of people, offering a range of different answers. I’ll add mine:
Yes, morality is a complex topic. But the current state of machine learning gives us a pretty simple answer to this question: No.
Nobody knows how to instill values or morals into AIs. We can tweak their behavior, but not the deep machinery that underlies it. In other words, an AI understanding a set of rules doesn’t mean it’ll choose to follow them. We’ve seen this time and again. To name a few examples:
AI models inducing psychosis even though, when asked, they affirm this behavior to be morally wrong
AIs providing instructions for cooking meth, manipulating the election, and a range of other nefarious tasks after jailbreaking (see pg. 15 for additional examples)
AIs refusing to be shut down, even when told, explicitly, not to refuse shutdown.
So I disagree with Andrew Rogoyski, from the Institute for People-Centred AI, who offered the view: “Perhaps the real question is whether we understand morality well enough to codify it.”
Even if we had moral consensus, the best we could do is communicate that consensus to the AI and hope it used our framework in the way we intended. This is quite different from programming in any kind of morality. In fact, if we could program in anything at all, that would be an incredible leap from the world we currently inhabit, where AIs are grown rather than programmed.
I feel similarly about Assistant Professor of International Law Jessica Dorsey’s concern that we need to determine whose morality the drone is following. To be fair, she’s using it in the narrower context of autonomous weapons, highlighting that there’s no global consensus on their governance. But the broader idea that an AI could embody the morality of a certain group fuels race dynamics that help justify the US’s light touch approach to AI acceleration. (Wouldn’t it be better for American-made AI to dominate and not Chinese-made AI? We better hurry up and build superintelligence first.)
Like I covered last week, a similar race dynamic seems to underlie justifications for going all-speed-ahead with autonomous warfare. From the article:
Some experts argue that giving drones greater autonomy, and programming rules of engagement and morality into them, will be a necessity if other nation states continue to develop and deploy similar technology at pace.
(Note the “program into” phrase again.)
Another view featured in the article comes from Alex Fink, the US chief executive of a US-Ukraine startup that makes software for autonomous drones. He seems to be pushing back on the need for autonomous judgement, proposing what I’d argue is a more dangerous approach. As summarized by the Guardian:
There could be a future scenario where humans might not select individual targets at all but instead designate a “kill box”, in which anything in a given area at a given time is fair game for an autonomous system to target.
“A human certifies this area, there are no friendlies, and I guarantee there will be no friendlies in the next 15 minutes. Any vehicle in this area is pre-approved for this period of time,” says Fink.
It seems to me like the worst of both worlds to give AIs blanket instructions like “Kill everything in this box no matter what” and not allow for new and relevant info to be taken into account. Arguably, you wouldn’t even need a particularly advanced intelligence to do this, provided it could distinguish between “already dead” and “still need to kill”. More importantly, though, judgment shouldn’t be taken out of the process.
To illustrate this, remember Stanislav Petrov, a man widely credited with saving the world from nuclear disaster when he went against protocol and decided not to report the US missile launch indications flagged by the computer system (later confirmed to be a false alarm) in 1983. If he had followed the blanket rule he was given to report all incoming missile detections to the Soviet military, there could easily have been nuclear war.
Scorsese criticized over AI storyboards
The BBC reported today on Martin Scorsese’s controversial endorsement of AI to instantly create storyboard images, which communicate the look of particular scenes. He called the tool “creatively freeing” and said it allowed him to “more clearly and efficiently” communicate his ideas to his production team, something the article notes “he had always struggled” to do.
The statement drew criticism from artists, who say there’s no reason to rely on AI for something human storyboard artists can already do. From the article:
Karla Ortiz, who worked in the art department on films including Avengers: Endgame, Black Panther and Doctor Strange, wrote on X: “He throws every single storyboard artist he’s ever worked with under the bus, as he demolishes their livelihoods with models that are likely trained on those storyboard artist’s same works.
“To use his legacy and power for this is just so disgusting.”
The BBC cites Darren Aronofsky, Steven Soderbergh, and Steven Spielberg as additional examples of directors who are starting to rely on AI for various aspects of the filmmaking process.
The controversy over AI’s role in the creative process and its potential to displace human artists is not new; I expect we’ll see a lot more coverage of this sort. Notable to me is the reporting: quotes from both Scorsese’s supporters and criticizers are taken straight from social media, highlighting how big of a role grassroots commentary is likely to play as these debates intensify.
Dispatch from Beck
Shifting responsible scaling policies
With the release of Claude Opus 4.8, Anthropic updated its policy for responsible AI development. They write:
“Our Responsible Scaling Policy (RSP) is our voluntary framework for managing catastrophic risks from advanced AI systems. It establishes how we identify and evaluate risks, how we make decisions about AI development and deployment, […] We will continually update the RSP as we learn more…”
Many other frontier AI companies have equivalents — OpenAI calls their similar document a Preparedness Framework. But they all identify potential risks related to the advancing capabilities of models. They focus on risk categories like AI-enabled cyber attacks, AI models engaging in recursive self improvement, and AI facilitation of the development of chemical, biological, or nuclear weapons. While the existence of RSPs is much preferable to firms simply ignoring the problems, companies update these documents readily when inconvenient, so it remains unclear to what extent an RSP reduces the threats of catastrophic harm.
Prior versions of Anthropic’s RSP had specific triggers for the company to change its security standards (so a model couldn’t be stolen or misused) when a model could help produce biological weapons. It committed to strengthening defenses when a model was capable of “significantly help[ing] threat actors.” But now these triggers have been significantly weakened to only occur when a model “functionally substitute[s] for scarce human expertise [...] that would otherwise require recruiting one of a small number of world-leading specialists.” That’s inarguably a weaker standard, and if some attack were to occur, I wouldn’t feel any better about it knowing that it didn’t take a world-class expert or AI equivalent.
Anthropic describes the change as a clarification in the system card (the most widely read report of model capabilities and testing), but within the RSP itself, it says the change “revises our threshold for novel chemical/biological weapons production to better track the threat model of concern.” I hope de-emphasizing the scale of changes (‘clarification’ instead of ‘revision’) doesn’t become a pattern — it’s hard to reconcile with the company’s claim that they are just updating these documents as they learn more.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.