Dispatch from Beck
Will there be jobs?
Sam Altman, OpenAI CEO, used to say that AI will “probably replace most of the jobs people do today.”
Dario Amodei, Anthropic CEO, has said that AI will replace half of white-collar work.
Mustafa Suleyman, Microsoft Chief Executive for AI, says that “all white collar jobs” will be automated in 18 months.
Jack Clark, Anthropic co-founder, said last week that the world is “in denial” about the capabilities of current AI models (and that AI has a “non-zero” chance of killing us all).
We’ve covered predictions of a jobs apocalypse before. Many dismiss these claims as hype, used to drive up company valuations. I’m skeptical that it’s hype, both on an object-level evaluation of capabilities and their trends (for more on the object-level trends, see the METR time horizons chart, including StopWatch coverage) and also because it’s obviously a bad strategy.
People don’t like it when you tell them that their jobs will end without a clear replacement. Pew Research reports that among the US public, “50% say they’re more concerned than excited about the increased use of AI in daily life, up from 37% in 2021.”
In these circumstances, with the companies’ stock about to become publicly traded, and facing negative public opinion, some are walking back their predictions. TIME reports that Altman has reversed his prior claims: he is “delighted to be wrong” about the “jobs apocalypse that some of the companies in our space advocate or talk about.” He said he thought there would already be more effects on entry-level employment and that his intuitions “were just off.” And Dario has changed his tune too, noting that he now thinks that if AI automates “90% of the job” then the 10% left will expand to become 100% of the job. Such claims seem inconsistent with his vision of a “country of geniuses in a datacenter.”
It is challenging to disentangle these evolving views from the incentives these executives face. Peter Wildeford, of the AI Policy Network, says, “It’s hard to say whether they’ve actually changed their forecasts for AI’s economic impact, or whether they’re just trying to change the narrative.”
Dispatches from Joe
VP Vance praises the encyclical
Alongside a flurry of news coverage, Pope Leo XIV’s new encyclical drew praise from U.S. Vice President JD Vance and criticism from former AI czar David Sacks.
The vice president called the pope’s words “profound, and the sort of thing that you would expect and hope from a leader of the church,” and added, “if we make it through [the AI age] successfully, it will be in large part because the pope and the church are able to provide the kind of moral leadership that we need.”
I notice a tension between what that leadership is saying and the tech-first agenda Vance himself often espouses. His remarks at last year’s AI Action Summit in Paris sounded deeply opposed to regulation of any kind: “The AI future is not going to be won by hand-wringing about safety.”
Having grown up deeply Catholic myself, I’m well aware that being a person of faith doesn’t mean deferring to the church on all matters of import. But I side with the pope on this one: for humanity to survive and flourish, the AI race can’t be allowed to continue. I hope Vance comes to understand this as well.
David Sacks is the former White House AI and crypto czar, and still commands significant influence from outside the administration. Last week, he was largely credited with convincing the president to cancel a planned executive order that would have created a voluntary review process for AI. Sacks can usually be relied upon to vehemently oppose AI regulation, and his commentary on the pope’s encyclical did not disappoint:
The Pope rightly warns that AI must serve human dignity, not become a tool of domination or exclusion.
But if we hand governments sweeping power over AI development in the name of safety, how do we prevent it from being used to censor, surveil, and control citizens — as Orwell foretold in 1984?
These comments strike me as not-even-wrong; they’re completely beside the point. Refusing to regulate AI development will not prevent governments from using developed AIs as they please, including for censorship, surveillance, and control. It’s an appalling situation, but that’s no reason to sit on our hands and let AI companies build yet more capable systems.
Like it or not, governments already have sweeping power over AI development, as they have sweeping power over many other things. Thus far, they’ve declined to use that power to put a halt to the AI race. Narrowly exercising that power against frontier AI development would contribute little more to authoritarian regimes than nuclear nonproliferation did.
Champion hacker expects to fall behind AI
For several years, Valentina “Chompie” Palmiotti has been winning awards for her extraordinary computer skills. But she worries her latest standout performance at an international hacking competition may also be her last.
Chompie uses AI tools to help her find bugs and backdoors, but she told the BBC that the field of hacking may soon eclipse her talents, thanks to AI models like Mythos and GPT 5.5 Cyber. The BBC also quoted Orange Tsai, a Taiwanese hacker, who is excited by how AI helps his workflow but hopes humans will remain forever advantaged.
I think Chompie has the right of it. As she pointed out to the BBC, hacking currently occupies a “sweet spot” in which AIs are extraordinarily useful tools. Today’s AI can’t do all the work itself; yesterday’s AI could barely do any. If the present capabilities race continues, tomorrow’s AI may well render even the best hackers, and subsequently everyone else, obsolete.
Dispatches from Alana
Chatbots and student writing
Brookings Institution education director Rebecca Winthrop argued today that AI “constricts our full range of thoughts and our ability to generate original and useful ideas” and that this phenomenon may be especially true for students.
She cites a few studies, leaning on Georgetown neuroscientist Adam Green’s analysis of 370,000 personal statements in college application essays. The study compared essays before and after ChatGPT became available, and concluded that the post-ChatGPT essays, in Winthrop’s words, “used diverse and colorful language, but lacked truly creative ideas.” Interestingly, human raters gave post-ChatGPT essays higher creativity scores.
On a quick skim of the study, it seems like most of these higher creativity ratings were actually done by an AI model. Human raters first rated a smaller subset of essays, and then the model was trained to imitate these. In the subset, word-level diversity was found to be a strong predictor of high creativity ratings, while distinctness at the document level was less important. In other words, raters seemed to view colorful, diverse language as creative while paying less attention to how unique the concepts were. Winthrop’s interpretation is “the linguistic coverup worked”; another might be differing definitions of creativity.
Winthrop argues that using LLMs for brainstorming might be more problematic than most think, because this is “the work that’s fundamental to writing.” She cautions against the homogenization and flattening of ideas.
While I haven’t taken a deep dive into the studies, I think there’s some nuance missing from Winthrop’s conclusion. As a former university writing center tutor, I’m a firm advocate of talking through your writing projects with someone (even a rubber duck) before you start. It often helps people formulate and organize their thoughts, remove blocks, and yes — generate ideas. If the ideas are fed directly to you by the bot, that’s a problem. But a real “pre-writing” conversation usually sparks thoughts in the writer’s mind that might not have surfaced without the conversation element. If students can use bots as thought partners whose feedback they take with a hefty grain of salt, I’d guess the effects would be mostly positive. (Though of course, that’s a big if.)
AI agents everywhere
Remember when OpenAI didn’t let their models access the live internet for safety reasons? That was only just starting to change in 2023.
Now, just three years later, it seems like a completely different world.
Reuters reported today that you can now use an AI agent to trade stocks and make credit card purchases on the brokerage platform Robinhood.
Reuters defines agents as “digital assistants that go beyond chatbot-style prompt responses by autonomously planning and making their own decisions,” and points out that “financial technology companies are racing to turn AI agents from experimental assistants into tools that can carry out real-world transactions.”
Indeed, AI agents such as those Robinhood now supports are becoming increasingly common. Reuters brings up Visa’s recently released (2025) platform where AI agents can shop online for you. Other examples that come to mind are trading crypto (at your own risk), letting them connect with other bots on AI social media, and of course, vibe coding.
According to Reuters, “businesses warn that agentic AI adoption is outpacing their ability to monitor it” though “Robinhood executives said they had put enough guardrails in place to counter concerns of agents going rogue.” According to an April Deloitte survey, just 21% of the information technology and business leaders surveyed think the orgs they lead “have a mature governance model in place for agentic AI.”
My take? I see the Robinhood example as just one more piece of evidence that society isn’t planning to be particularly cautious. If relatively limited AI models are already embedded into every facet of society without mature governance, what happens when labs succeed at their goal of building smarter-than-human AI? Given frontier systems are currently black boxes that we don’t have a reliable way to steer, takeover scenarios (whether by bad actors or autonomous AI systems pursuing weird goals) become even easier.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.