"Just something doomers wanted."
Executive (dis)order, California labor, battlefield AI, and more
Dispatch from Beck
Executive fractures
New details and conflicting narratives are emerging following Trump’s decision to delay his executive order on AI. As we reported yesterday, Trump had intended to sign the order creating a voluntary review process for frontier AI models, but changed his mind at the last minute. He was focused on the race, telling reporters “We’re leading China, we’re leading everybody, and I don’t want to do anything that’s going to get in the way of that lead.”
But different sources hold different frames. Politico emphasizes the role of David Sacks, former White House AI czar. Between Wednesday night and Thursday morning, he “voiced industry concerns” to the president, including about how a 90-day preview period could hurt US companies’ lead, particularly if it became mandatory at some future point. But the same reporting also includes more mundane explanations from the White House — many invited tech CEOs couldn’t make it to the announcement on short notice. This explanation serves to downplay how strong the disagreements are and emphasizes the future possibility of an order.
Axios framed the postponement as the result of “anti-‘Doomer’ feedback.” One of their sources said, “The whole thing was unnecessary” and “just something doomers wanted.” They note that leading companies already have voluntary testing agreements through the Center for AI Standards and Innovation, and that different sources contest which departments should be involved in AI regulation.
This frames the choice to not issue the order as a win for tech accelerationists, and all regulatory effort as inherently “doomer,” a term meant to minimize those concerned about really bad outcomes from AI, including extinction. While I try not to be overly critical of others’ word choices, “doomer” once pointed to specific concerns; these days, it’s applied to anyone having any concerns whatsoever.
Treasury Secretary Scott Bessent had been an important part of coordinating the executive order, but not all parties to the negotiations approved of having him in this role. One tech source told Axios, “It’s not clear, just objectively speaking, why Treasury is involved and what is their substantive expertise in this area.” The Department of Commerce has also been a potential locus of control; it recently announced agreements with frontier labs, but then pulled the announcement from their website.
It seems that competing factions within the executive branch are elbowing for control of both AI regulation and the narrative. Even as mild as this executive order was proposed to be, it would be good for companies and government to get some practice in on these relatively easy questions. They’ll need it to address extinction.
Dispatches from Joe
Amateurs using AI generate sloppy code
Christopher Mims of the Wall Street Journal shares a warning from two engineers who built the core of OpenClaw, software designed to help an AI agent autonomously handle computer tasks. Widespread use of AI to “vibe code”, the engineers say, is causing a surge of “vibe slop” — low-quality, buggy code underpinning new and updated software. I’d consider these folks experts in buggy code; OpenClaw itself is famously insecure.
Jabs aside, how can it be the case that AI is both a terrible code-writer and a dangerously prolific hacker? There’s a mix of reasons, and I’ll name only a few. First, not all AIs are created equal. State of the art models are significantly better coders than free and open source AI. Second, since AI isn’t fully autonomous yet, the user still needs to understand some things about the big picture and ask for the right things. And third, it’s often easier to break a system than to make it robust. Hackers using AI can try thousands of attacks, and only one needs to work.
In any case, we’re seeing evidence the quality of public code is worsening. GitHub, the world’s main repository of openly shared code, had to upgrade its policies to help project maintainers counter a deluge of low-quality submissions.
What does this mean for you, the reader? Frankly, I wouldn’t let it stop you from trying out coding agents. They can be genuinely helpful on personal projects, and it’s useful to develop a basic feel for what they can do. If you’re concerned about security, you can in fact just ask your AI, and it’ll give pretty solid advice. I wouldn’t stress over code quality; as long as you aren’t downloading random files from untrusted websites or running your startup’s payroll on vibe-coded software, it’ll probably be fine. (If you are trying to run a startup, AI can still help, but you need to be more careful.)
Whatever you do, don’t run OpenClaw on your personal machine. It’s a trap.
“It’s not a plan, it’s a study”
California governor Gavin Newsom issued a new executive order on AI and labor. According to Politico, union organizers are unimpressed. Labor leaders originally offered conditional support for Newsom’s campaign if he met their demands on AI policy, but many now question whether he’s delivered.
The order asks the state labor agency to consult unions on collective bargaining around AI, to spin up a dashboard on how AI is affecting the state’s employment, and to field proposals for minimizing harm to workers.
“It’s not a plan, it’s a study,” complained one labor leader. They have a point; the California executive order is thick with “options and recommendations” but thin on precise action, and it exerts no restraint whatsoever on the behavior of California AI companies. Gathering information is perhaps an important step forward, but it’s not nearly enough.
OpenAI’s top lobbyist Chris Lehane, however, seems pleased. Politico quotes him as excited about Newsom’s order, and cites an OpenAI policy blueprint from April. This follows a separate interview with Lehane by Politico’s Brendan Bordelon (5/20), outlining OpenAI’s new political strategy of “reverse federalism”: promoting industry-favored policies at the state level in the hopes that they will become a de facto national standard.
Basically, while Congress avoids passing AI regulation of its own, it falls to the states to decide what rules to set. Previously, AI industry lobbyists have tried and failed to get the federal government to preempt state AI regulation, but the millions they spent in opposition won some success in blocking state bills or watering them down.
Lehane’s strategy looks to me like a tactical retreat. The industry failed to lock in favorable rules at the federal level, so now they’re trying the same thing in the states. If they can get some nice-sounding but ultimately toothless policies in place and then cement them as the standard to beat, perhaps they can forestall further unwanted interference. It’s a transparent attempt by industry to set the ceiling where the floor should be.
Europe depends on U.S. battlefield AI
Politico EU’s Antoaneta Roussi relays a worried report by a top NATO commander, Admiral Pierre Vandier: Europe has no good alternative to U.S. military AI. Concluding an uncommonly fast six-month deal, NATO has just installed Palantir’s Maven Smart System at command centers in Belgium and the Netherlands.
Europe may be worried about overreliance on U.S. tech, but I see a larger concern. I worked eight years as a reliability engineer in oil & gas, and the dependence of both NATO and U.S. militaries on a single supplier brings to mind a long-dreaded phrase: single point of failure. When many supposedly independent systems rely on the same tools or bottlenecks, they run a risk of simultaneous or correlated failure. We saw an example of this in 2024, when security giant CrowdStrike pushed an update that temporarily broke millions of Windows PCs, including at hospitals and airlines.
There are advantages to centralizing and standardizing systems, of course. It makes information sharing easier and helps different systems stay compatible, among other things. But there’s a genuine risk to standardization, and compounding that risk is the fact that Maven plugs into a suite of modern black-box AIs. I think the greatest risk to humanity comes from the AI companies themselves, but I still feel uncomfortable at the idea that the destructive potential of the U.S. and NATO military routes through systems so poorly understood.
Dispatches from Donald
New AI policy at UC Berkeley Law
Forbes’ Michael T. Nietzel reports on new AI rules adopted by UC Berkeley Law. Generative AI can be used to identify sources but is otherwise prohibited. Specifically banned uses include conceptualizing, translating between languages, use during exams, and uploading course materials to AI systems.
The policy change is due to the “rapid increase in capabilities in Claude,” according to Chris Hoofnagle, Faculty Director of the Berkeley Center for Law & Technology. The school acknowledges that future lawyers will need to be fluent with AI – Nietzel points out that many lawyers use AI even today – but believes that students must first develop their own skills. “We don’t want students to write the best possible paper, but rather the best possible paper that the student is capable of,” Hoofnagle said.
There are a lot of AI policies at a lot of schools. I think this is interesting because UC Berkeley Law isn’t denying the usefulness of AI, but rather affirming the risk of cognitive atrophy. Early in StopWatch’s history, the team discussed the role that AI would play on this site. We committed to writing the dispatches by hand.
Using AI would be a lot easier, but I’m happy to be on a team that’s made that commitment. There are a few reasons that I, personally, prefer human-driven work. In this case, it’s not because our AI tools can’t do the job well, but because they can. But the skills that I exercise at StopWatch seem holistic: If I handed eight of ten tasks to the AI and reserved the last two for myself, I think that I might lose my edge not just on the eight but on the other two as well.
HHS to use ChatGPT in audits
The Associated Press’s Ali Swenson reports that the U.S. Department of Health and Human Services will expand its use of ChatGPT and other AI tools to analyze audits of state Medicaid and other HHS-funded programs.
The administration characterizes this as a useful tool in cracking down on fraud. Critics note that AI tools still make frequent errors. Just last month Swenson reported on a fraud probe into New York’s Medicaid program: Roughly 5 million people were alleged to be receiving personal care services, when in reality the number is about 450,000.
AI was not responsible for the misreported figure, but introducing AI tools to this kind of work creates new opportunities for costly errors. My worries about AI are not limited to what future models will be able to do. AI can be dangerous because it does a good job on bad things, like making illicit copies of itself, but it can also be dangerous because it does a bad job on good things. The current models can be dangerously unreliable; some people are relying on them anyway.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.