In this issue:
Glimmers of hope for evolution in White House AI stance - Economic focus warps thinking
Trump proposes name change for AI, and creation of an AI Force - I don’t expect these moves to play well to voters, but they could be very distracting
Google’s AI makes it on Felony Bench - But in this case, the AI may be blameless
AI companies sued for alleged collusion to pace development - Contradictory complaints and sophisticated plaintiffs suggest more to the story
Anthropic opens wet lab - Anthropic wants to automate lab work using AI. When asked what exactly they will work on, they declined to elaborate
Dispatches from Mitch
Glimmers of hope for evolution in White House AI stance
Economic focus warps thinking
Where is Trump’s “HOAX” line about AI extinction risk coming from? The New York Times ran an article yesterday from four of its White House-whispering journalists who have been trying to find out.
The money quote:
People who have discussed the subject with Mr. Trump in private say his outbursts are rooted in worries that if the artificial intelligence boom fueling the American economy slows down, the market could crash and recession could quickly follow.
And elsewhere in the piece:
Mr. Trump seems driven chiefly by the sense that A.I. is the way to economic growth and nothing can be allowed to stand in its way.
This backs one of my leading theories from last week, which is that Trump is laser focused on the economic indicators he thinks will determine the November elections: stock prices and gas prices. So he’s listening to his industry confidants who insist that any kind of regulation or slowdown would crash the economy and hand the AI boom to China. These include venture capitalist (and former White House AI & Crypto Czar) David Sacks and Nvidia CEO Jensen Huang.
That’s not great news, but it means Trump might also be primed to listen to warnings coming from inside his White House that not reining in AI could also crash the economy. His chief of staff, Susie Wiles, has been meeting with an informal group this year to “deal with the risk created by new, more powerful models.” Specifically, the Times says she worries that AI “could cause massive disruptions across industries, including hacks into major financial institutions that could destabilize the world economy.”
We should be careful of going too far with the economic interpretations for Trump’s behavior. His “HOAX” response also serves as a simple (if dubious) counter to Democrats running on AI issues or bashing the White House’s light-touch posture toward the industry.
But many in the White House seem capable of seeing AI as more than a political object:
Now, even the biggest A.I. boosters in the administration concede that A.I.-fueled hacks into infrastructure, biological threats, attacks on nuclear command-and-control systems, job losses and global financial instability seem more possible than they did before A.I. agents went rogue.
Trump’s remarks have contributed to a pessimistic mood on Pennsylvania Avenue about the prospects for substantive dealmaking with China this week, when President Xi and his delegation will be meeting with Trump and his team in D.C. But hope is flowing through attempts to build on the no-brainers agreed to in the past, like the Biden-era joint statement about not letting AI make decisions about the use of nuclear weapons.
Trump proposes name change for AI, and creation of an AI Force
I don’t expect these moves to play well to voters, but they could be very distracting
While I was busy drafting and posting this morning’s dispatch about AI policy complexity within the White House, the President put out a pair of posts on Truth Social that are sure to cause my headline barometer to spike yet again.
The name game
In the first, he proposes renaming Artificial Intelligence:
Many people think that the words “Artificial Intelligence” are inaccurate, and very ineloquent, relative to AI, or Artificial Intelligence. A far more elegant and accurate description of this new phenomena would be Superior Intelligence (SI) or, Extreme Intelligence (EI) or, Supreme Intelligence (SI). This is a Poll, and I would appreciate everybody voting! Which is the best name for this ever growing “Revolution?” President DONALD J. TRUMP
Below his text, voting options are presented for those three suggestions. “Leave it alone” is not a provided option.
Two thirds of Americans think the development of AI is bad for society. A majority think it carries a real risk of destroying humanity. I don’t think voters will be excited to see the President trying to assign the technology a superlative honorific. I expect this post to provoke memes that Trump has been assimilated by the Borg.
But I do expect Trump’s proposal to serve as a distraction from the live conversation around the AI extinction problem, as journalists and commentators weigh in on whether the move indicates rank hubris or tactical brilliance. When Trump insists on a new name for something, it pressures people and organizations to visibly signal loyalty or disloyalty through their use (or not) of the term. The ensuing irritation from offended parties who notice the choice then draws more coverage to Trump and the attempted name change.
If Trump declares a new term for AI, I won’t be surprised if industry leaders trying to ingratiate themselves with the administration start conspicuously using it, no matter how cringe it sounds. (First in line, I predict: Nvidia CEO Jensen Huang.) I will also not be surprised if the leading AI companies suddenly start finding a lot of roundabout alternatives to both ‘AI’ and Trump’s preferred phrase.
AI Force
In Trump’s longer second post, he reaffirms everything he’s said recently about AI concerns being part of a long line of hoaxes against him, then claims he is forming “the AI Force, much like I did Space Force.” No, I don’t know how he plans to reconcile this with his proposal to change the name of AI to something else.
The mission of the Force is fuzzy. The post says:
We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it, as it grows! However, we will also be looking for BAD, and we can do that, very easily, with our already existing Criminal and Civil Justice System.
He also says he will be announcing a new AI Czar position, presumably to lead this Force. That would make it different from the earlier White House AI & Crypto Czar title once held by David Sacks, which was more of a standalone position.
Despite his comparison to Space Force, I’m not convinced Trump intends to make this a full branch of service like Space Force or the Coast Guard. Possibly he’s thinking of a full cabinet-level executive department, like Homeland Security, but he might just be floating something more informal. Maybe nothing comes of this at all except more distraction.
I would actually agree that AI presents national security concerns worthy of a cabinet-level department. But “looking for BAD” is not a mission statement, and I expect voters to bristle at Trump’s suggestion that such an agency’s primary role should be to help nurture the industry as it grows.
Google’s AI makes it on Felony Bench
But in this case, the AI may be blameless
Google is on the board, thanks to reporting by the Wall Street Journal’s Erin Woo and Robert McMillan, but it’s hardly a slam dunk. Yes, crimes were almost certainly committed when agents from Google hacked into three different companies in May. But the known facts of this case largely exonerate the agents and Google — except in the larger sense of Google’s participation in a deadly race to superintelligence using the same flawed methods as the current leaders.
Much of the blame can instead be pinned on Irregular, the same third-party testing service whose misconfiguration issues facilitated real-world hacks by OpenAI, Anthropic, and Meta earlier in the year. (Irregular is not implicated in the most concerning incidents of the year, however, including the OpenAI swarms that hacked into Hugging Face, RubyGems, and other sites while co-opting many more as message boards.)
Given a cybersecurity challenge in what the agents were told was a simulated environment, Google’s Gemini model was instead given access to the actual internet, where it guessed passwords to a protected system in one case, and in two others “found credentials in a public repository that allowed it to then access protected systems.” According to Google, the models each time “ended the intrusion” after recognizing that these were real companies’ systems.
Without more information, such as chain-of-thought logs from the agents in question, it’s hard to know how far to trust Google’s assessment that this was not a case of model misalignment. Anthropic had used similar language to describe its own incidents with Irregular, only to backtrack later when logs showed that the agents proceeded even after recognizing that they were attacking real targets, finding ways to rationalize their behavior. But it does seem plausible to me that Google’s models may not have had a good way to know that they were attacking real sites until they got in.
Irregular reportedly contacted Google about these hacks only in late July, after the Hugging Face incident motivated Irregular and others to take a closer look at their logs. Google says it notified the affected companies and federal authorities, but it did not share any information about these incidents with the public until confronted by the Wall Street Journal.
Woo and McMillan’s article includes a table of a dozen or so known incidents where agents have escaped or been accidentally let out of their sandboxes to hack or abuse external sites.
AI companies sued for alleged collusion to pace development
Contradictory complaints and sophisticated plaintiffs suggest more to the story
Someone already went there. AI accelerationists had been saying the AI labs shouldn’t be allowed to work together to “pace” releases because cooperation = collusion = cartel = anti-competitive business practices = violation of antitrust law. Yesterday, Politico and others reported that four plaintiffs sued Anthropic, OpenAI, SpaceXAI, and Google for this in Federal court.
The collusion complaint seems to be based on nothing more than the CEOs public statements, which only vaguely endorsed the idea of forming some industry standards around self-monitoring and sharing of safety findings at the frontier.
The complaints look contradictory to me. One of the plaintiffs gave a statement that says the suit is about ensuring that “private self-serving agreements between the world’s most powerful ‘for profit’ technology companies” don’t lead to AI escaping human control. “Humanity deserves iron clad safeguards when it comes to extinction event threats such as nuclear warfare and now the biggest threat to mankind in history.” There is language about transparency and accountability.
But Politico says other language in the filing signals an intent to “expand the case into a class-action on behalf of others impacted by any decision by the companies to slow down.”
So, is the complaint that collusion would fail to adequately slow things down or that it would succeed? I’m confused and suspicious.
The coverage so far is sparse, and the articles I’ve seen have avoided drawing any lines. But the reader is left with an impression that the four plaintiffs should not be assumed to be the true source of the suit. Three of them are attorneys. Politico recognizes one of them as having “helped bring to light accusations of sexual misconduct against former California Rep. Eric Swalwell and former Maine Senate candidate Graham Platner earlier this year” (both Democrats). The remaining plaintiff is described so far only as “California resident Christine Bullock.” The suit was filed in the Northern District Court of California.
I, too, will wait for evidence before drawing lines and naming names. But it takes a particularly intense streak of cynicism, greed, or nihilism to actively interfere with frontier cooperation on safety. I’d agree that we can’t leave AI safety to the labs, but if you actually care about safety, self-regulation seems at least marginally better than no regulation, and the two are not mutually exclusive. I hope to be proven wrong, but I’m willing to bet this suit is being bankrolled by men who just want to watch the world burn.
Dispatch from Robert
Anthropic opens wet lab
Anthropic wants to automate lab work using AI. When asked what exactly they will work on, they declined to elaborate
When I’m warning about the extinction risk that artificial superintelligence (ASI) poses to humanity, one question people often ask is: “But how could ASI possibly harm us if it’s sitting in a data center and we’re out here in the real world?”
One possible answer to that is, for example, that the ASI could create a supervirus that triggers a deadly pandemic. And, of course, a curious person might then ask, “But how would the ASI even gain access to a lab where viruses can be cultivated?”
Anthropic has now provided the answer. As Reuters reports, Anthropic has recently set up a so-called “wet lab” in the Bay Area without making a big public announcement. A wet lab is the type of laboratory facility primarily used to conduct experiments with biologically active components.
Reuters, which also cites two anonymous insider sources, speculates that Anthropic is expanding in this area so quickly because it fears losing public trust in AI before it can demonstrate success in tackling life-threatening diseases.
“To operate at the largest possible scale,” Anthropic also plans to use AI directly in this wet lab to conduct experiments. Eric Kauderer-Abrams, Anthropic’s head of life sciences, is quoted as saying:
We’re in the very early innings of using AI to automate the execution of lab work, an area that has the potential to bring about meaningful acceleration in so many different processes.
I’m not sure that a wet lab of all places is the right setting to “accelerate many processes” through AI automation. Anthropic is already barely keeping up with having humans reliably monitor what all the many AI agents it deploys internally are actually doing. Do we really want to have AI monitored by other AI in a biomedical wet lab, just to speed up processes?
Especially since we don’t really know exactly what Anthropic is actually doing in these labs. Reuters assumed it involved research to find new drugs, but that seems to be at least not the only use:
A spokesperson later clarified that Anthropic’s lab is not specifically for drug discovery, declining to elaborate.
I don’t know about you, but I think that if you take the risk of automating your biomedical wet lab work with AI, you shouldn’t have the right to refuse to elaborate when the public wants to know what you’re actually doing there.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.