(Apologies for an earlier email that contained just the first dispatch.)
In this issue:
OpenAI feels heat from widening swarm scandal - Corporate spin machine goes into overdrive
AI destroys essay-mill economy - Is your industry next?
Could Europe stop the AI race on its own? - No, but it could slow things considerably
Dispatches from Mitch
OpenAI feels heat from widening swarm scandal
Corporate spin machine goes into overdrive
California’s Attorney General is the latest to join the more-than-a-dozen states investigating OpenAI over the Hugging Face swarm incident, Politico reports.
Meanwhile, the company put out a statement about the German wiki found to have been co-opted by a swarm of 3,000 OpenAI agents. The agents appeared to be coordinating on general knowledge evaluations, and were exploring whether they would get terminated after completing a fifth research task.
OpenAI’s statement is notable for its hollowness. It acknowledges the incident, saying that “our agents wrote to several internet sites,” but admits no fault and provides no details. It makes no mention of new reporting that some of the co-opted sites belonged to colleges, and one to a high school AP chemistry class.
The company disingenuously claims the wiki incident was already acknowledged by earlier papers going back as far as March:
Prior to the Hugging Face incident, we saw early signs of agents using the internet in unintended ways, as reported in [these, earlier, posts]. We considered the wiki incident to be an instance of misalignment similar to the ones we’d shared.
The pull-quote turned into an accompanying image just says: “We’re working on a framework for when and how we share AI misalignment incidents.”
I increasingly worry there might be a connection between OpenAI’s caginess about its swarm incidents and the brazen recklessness of its new model release. Maybe the scandal is deep enough to destroy the company. Maybe OpenAI isn’t just racing to beat its competitors to superintelligence, but to outrun the investigations.
I hope the attorneys general and members of Congress looking into the incidents move quickly. I hope they understand that OpenAI’s executives might be rushing to amass AI capabilities they hope will put them above the law but would just get everyone killed.
AI destroys essay-mill economy
Is your industry next?
For me, the key insight was obvious from the headline: “Kenyans Made a Living Writing College Essays. Then A.I. Arrived.” It hadn’t occurred to me that these folks would be out of work, but of course they would be. AI’s grip on the outsource-your-homework market is near total.
No matter how unsympathetic we find these victims, we should reckon with the implications of their experience, because there’s a lot of contradictory talk about whether AI is creating more jobs than it destroys or vice versa.
I think the most important observation is that the Jevons Paradox applied, but it didn’t save the Kenyans’ jobs. I last wrote about Jevons in May. If you need the refresher:
In the early 1860s, economist William Stanley Jevons argued that demand for coal in England wasn’t going to fall as a result of efficiency gains in engines and furnaces. Instead, he predicted the opposite: that by making coal a more efficient fuel, these advancements would make coal power viable for many more uses, greatly increasing demand. He was right.
As schoolwork completion processes have become more efficient, demand for this product has indeed gone through the roof. There’s more cheating than ever before from students who never would have paid $40 to $70 for an artisanal, hand-crafted, Made-in-Kenya paper.
But this hasn’t created more work than ever for Kenyans managing fleets of chatbots. It has cut them out of the loop entirely. (The article says that the little essay work left in Kenya is for humanizers who tweak papers to evade detection tools.)
This is a problem for the popular argument “You won’t lose your job to AI, but to a human who learns to use AI.” That argument relies on Jevons, and the fall of the essay-mill industry shows us that Jevons only benefits workers occupying a bottleneck that demand must pass through. Ghostwriters hold no such terrain. Neither do the justifiably nervous mathematicians, insurance claims adjusters, entertainers, and content creators.
There are stronger bottlenecks in fields with legal moats. Where only a select few are allowed to sign off on the work, we may indeed see humans orchestrating fleets of agents for at least a short time: medicine, law, civil engineering.
This is small comfort. If the AI race isn’t halted, anyone who isn’t already out of work probably ends up dead. But if the race is halted such that models even a little bit stronger than the current state-of-the-art remain legal, then I think most people still end up out of work. You only need so many humans atop accountability pyramids, and demand for services provided by fields with legal moats is finite.
We also have to consider the enormous pressure the AI industry would apply to get its technology approved for the licensed work that remains. The very gatekeeping that insulates licensed jobs creates upsetting shortages for their services, driving up costs; that’s often the whole point. There will be no running from the moral argument that an AI doctor statistically better than a human doctor, accessible 24/7 to the most rural and least privileged, should be allowed to practice medicine.
In survivable futures with significant AI job destruction, wealth distribution programs will probably be needed to prevent social upheaval. This is true even if we’re not ready to start writing checks to people who used to write Johnny’s essays.
Dispatch from Donald
Could Europe stop the AI race on its own?
No, but it could slow things considerably
Writing in The Guardian, Alexander Hurst argues that the EU should ban exports of extreme ultraviolet (EUV) lithography machines, which are used to produce the most advanced computer chips in the world. The AI industry’s dependence on those chips — and therefore these machines — is absolute, and just one company in the world, the Netherlands’ ASML, builds the machines that make those chips. (China is attempting to reverse-engineer the technology, but they are not there yet.) As my colleague Mitch noted previously, they are very big — the size of a bus — and very rare: In June ASML counted 314 EUV machines in operation worldwide, with another 26 decommissioned.
All machines have a limited lifespan. This is true of the computer chips that the frontier labs use to train more powerful AI models, and true of the fabrication equipment that makes those chips. The more intensively the frontier labs use their chips, the more quickly those chips have to be replaced. Sufficiently strict export controls would force the AI industry to “tread water, to jog in place,” as Hurst puts it — giving the rest of the world some breathing room. Regulators could have a hope of catching up. But how much breathing room, and how much hope, depend on the strictness of those controls.
If you simply ban the sale of EUV machines, then you’ve put a cap on production capacity. That will have a real effect, but slowly, in an industry whose pace of change is one of the chief reasons to ban exports. The fabs that produce the chips already have EUV machines, and by ASML’s own account, “almost every lithography system that we’ve ever shipped is still in use at a customer fab.” (One of its earliest lithography machines, the PAS 5500 model, was released in 1991 and remains in service thirty-five years later.) It’s less like putting out a fire in your house than sending away the people who are pouring gasoline on your lawn; you’ve just spared yourself one future headache.
However, export controls can extend from hardware and other goods to the technical assistance that is required to maintain them. An update in 2025 added advanced lithography equipment to the European Union’s regulation on dual-use export controls, which governs that technical assistance. And do EUV machines require technical assistance? Loads and loads of it. In 2024, when the United States was pressuring ASML to restrict maintenance of machines operated by Chinese customers, a commentator quoted by Asia Times stated, “that equipment will become useless once it has any minor problem.” And these were only deep-ultraviolet (DUV) lithography machines — incredibly sophisticated, to be sure, but markedly less so, and less demanding of maintenance, than the EUV machines on which the frontier labs depend.
This doesn’t do anything to the chips that exist, but the chips that will exist in the future (or rather the chips that may exist, that a ban could preempt) are crucial to the frontier labs and their partners. Each training run is made with the assumption that there will be sufficient chips for the next training run to be even bigger, and every hyperscaler is building datacenters on the assumption that chips will arrive to fill them. The semiconductor manufacturer TSMC doubled its historical pace of construction in 2025 and 2026; its chief executive told shareholders in June that “it will be a long time before we can meet customer demand.” But add one thorough ban, let simmer, and subtract year upon year of planned-for chips — and those floating castles that the industry built up in its collective mind become mere rough-shaped clouds: The buildouts stall and compute can no longer be treated as cheap, because the most advanced chips simply cannot be replaced.
It’s a sound proposal. I mean, it’s no international treaty to ban the development of artificial superintelligence, but it could have a near-immediate impact. My praise comes with a few caveats, though.
There’s a possible short-term downside: Regulations don’t usually take effect as soon as they’re proposed. As soon as regulators look serious about a ban, chip manufacturers will try to surge production; and the frontier labs may be tempted to sprint — true, the most advanced chips can’t be replaced, but they can’t sit on a shelf indefinitely, either.
Cutting off the chip supply won’t freeze progress on its own, because recent capability advancements have come partly through improvements in techniques like post-training. Progress will slow, but not be halted. If export controls are the tool that’s used to sever the supply chain, the natural response will be to relocate chip manufacturing to Europe. That process could take a couple of years, even on the low end, though. Chip manufacturing requires other infrastructure and specialized personnel.
Regulations can move slowly, too, but there’s a salve for this: The Netherlands can take action first, while slower political processes go to work in the European Union. The Netherlands has historically handled ASML extremely lightly, but historically, the world was a safer place. Last year, AI agents were not breaking out of test environments and autonomously performing cyberattacks. Next year, who knows — and this year, there’s still time left for the Dutch to do something.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.