Retrofits
Shrinking timelines, lawslop in Congress, and AI treaty verification technology
In this issue:
Timelines shrink in AI Futures quarterly update - Three new methods give the same unnerving answers
Congressional office drowning in lawslop - Congress’s legislative proofreaders say AI-generated bills aren’t ready for primetime
Verifiers verified - The tech needed to verify an AI pause or slowdown treaty is coming along, but is shamefully under-resourced
Dispatches from Mitch
Timelines shrink in AI Futures quarterly update
Three new methods give the same unnerving answers
AI Futures Project, the team behind the AI 2027 forecast and AI 2040: Plan A scenario, just put out its quarterly forecast update. The headline figures:
Reality seems to be progressing about 75% as quickly as the original AI 2027 scenario (so still very fast).
Daniel Kokotajlo’s median prediction for the arrival of artificial superintelligence (ASI) is now 2029, vs. 2030 in the previous quarterly update. Eli Lifland now predicts this for 2033, vs. 2035 from the previous update. Their definition of ASI seems reasonable to me: “the gap between an ASI and the best humans is 2x greater than the gap between the best humans and the median professional, at virtually all cognitive tasks.”
The team now specifies that their forecasts assume AI development continues full speed ahead; a government-imposed slowdown would change things.
While they never claim high confidence in their forecast dates, they say that three different new methods they’ve devised for tracking the rate of progress are all, to their surprise, giving very similar answers.
These methods are interesting. One involves looking at Anthropic’s employee surveys that ask how much of a speedup at coding its developers think they’re getting. While AI Futures agrees with Anthropic that these figures are probably inflated, they’re going up — doubling about every 3.5 months, after some adjustment — and it seems very unlikely that employees’ tendency to overestimate is growing at the same rate.
An intermediate milestone that this is used to estimate is what the team calls the “automated coder” (AC): the point where “the leading AI company would rather fire its human software engineers than forego AI usage for coding.” Kokotajlo’s estimated date for this is now November 2027, vs. May 2028 in the previous update.
Reflecting on AI 2027, published in April 2025, the authors note things seem mostly on target, but they flag a few of their misses. One of those misses is probably good news: They originally expected that Chinese leadership would have “woken up” to AI by now and started unifying its leading labs’ efforts to match or exceed America’s frontier models. But while the world in general is more “awake,” the Chinese Communist Party still seems more interested in spreading present AI benefits across its economy than in racing to superintelligence, leaving each of its domestic labs to pursue ASI (or not) as they see fit.
Another miss, the team’s claimed “biggest predictive error,” is probably bad news: They expected greater public salience of AI by now. Growing awareness of the proximity to ASI and the extreme stakes this entails could make an imposed global slowdown more likely. But while the team thinks public salience is definitely higher this year than last, their chosen metric for this has barely budged, and awareness is not as high as it probably needs to be.
AI 2027’s forecast that the best Chinese models would be about 7 months behind the best American models by now seems about right to its authors (and to me). But in another bad-news miss, the team assesses that AI companies haven’t yet reached the expected degree of security with regard to their model weights and other secrets. The forecast expects serious efforts by nation-states to steal these. This year’s string of leaks and autonomous breakouts isn’t what anyone would expect from labs positioned to thwart nation-state intrusions.
Congressional office drowning in lawslop
Congress’s legislative proofreaders say AI-generated bills aren’t ready for primetime
Just a few days ago, my colleague Alana covered the widespread use of AI in U.S. congressional offices. She speculated that it could speed up the process of drafting legislation, among other things.
Politico’s Owen Dahlkamp confirms this today in an article about conditions inside the Office of the Legislative Counsel (OLC), which provides drafting services to committees and House members. The OLC does important work making legally sound text out of rough proposals and amateur copy. But according to “eight current and former officials who work or have worked in the office,” the deluge of AI-written drafts pouring in from congressional offices and outside groups is full of mistakes and ambiguities. These slow the legislative process and can lead to lawsuits if passed into law.
As a simple example of the kind of thing the OLC often has to fix, a bill might define a state in a way that only applies to the 50 states, inadvertently excluding Washington, D.C. and tribal nations. Today’s AI-drafted bills are also said to often lack nuance, as with whether funding for a program should come via a “tax credit, tax deduction, tax exclusion or a grant.”
Requests for OLC services in the first 60 days of the current Congress are up 72 percent over the same period two years ago. The risk seen by House veterans is that members unwilling to wait for the services of a bogged-down OLC may introduce raw AI-generated bill text. So far, at least, “No bills that have been voted on by either chamber have been shown to be entirely crafted by AI.”
The OLC is investigating an obvious partial fix: using AI tools as part of the legislative proofreading process. They are already well-suited to verifying that laws cited in draft legislation are accurately represented, and not the hallucinations of another AI.
No one quoted in the article mentions any concern that present or future AIs might slant legislative outcomes according to their own preferences, whether through unintentional bias or Machiavellian intent. I don’t expect legal maneuvering to play a central role in an AI takeover, but I do expect power-seeking AIs to deliberately try to bias laws in ways that would further their plans, as part of keeping their options open and greasing the gears of a more technological takeover. If we make superintelligent competitors for our planet’s resources, they won’t limit themselves to one lane.
Verifiers verified
The tech needed to verify an AI pause or slowdown treaty is coming along, but is shamefully under-resourced
An article in TIME by Billy Perrigo today provides a hopeful profile of a company working on a niche technology that may help save the world, but mostly encourages me as evidence that journalists are chasing the right stories.
Last month, more than a thousand employees of frontier AI companies declared that they wanted the government to work with industry to build the governance mechanisms needed to make a global AI slowdown viable. In our coverage, I pointed out that there has already been considerable research in this direction, but Perrigo’s article is the first I’ve seen to actually look into the supporting technology.
The company he profiles is Amodo Design, based in Sheffield, England. With mostly academic and non-profit funding, it is working on an AI verification system that would allow data center operators to prove their chips are only being used to run existing AI models rather than train new ones, and even to prove which model it is running.
The verifier is a pair of systems, one at the data center, and one for a monitor. The monitor’s version samples and reruns data fragments from the data center’s version, confirming that the AI being run is the one claimed.
The system still needs work to handle encrypted data, and currently requires between one-fifth and one-third the computing power of the AI model it is monitoring. And like most verification systems in development, it would require retrofitting existing data centers. But future iterations are expected to be much more efficient, and retrofitting would be a very surmountable obstacle for governments that start treating the race to superintelligence with the same seriousness as nuclear arms control. Arms treaties, the piece points out, have made use of monitoring technologies “like satellites and seismometers.”
This field looks shamefully under-resourced. Amodo’s CEO Tom Milton estimates that fewer than 50 engineers in the world are working full-time on verification technology. He employs 9 of them. “It is insane for us to think that we are even a noteworthy participant in this, let alone one of the largest projects.”
Perrigo concludes his piece by looking at whether an AI slowdown agreement is politically possible. Quoting an August recommendation from the Institute for Progress, he writes: “Better verification technology would unlock a broader space of possible agreements.”
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.