Congress seems to be waking up to the need to take action on advanced AI. While nobody else has yet joined Sanders in explicitly calling for a pause, Congresspeople are increasingly recognizing AI risks and governance challenges as worthy of serious attention.
Last week, Senator Lisa Blunt Rochester (D-Del) wrote letters to each of the three major AI companies, expressing concern and pressing them for more information about the incidents where their models broke containment and committed cyberattacks. Her letters characterized the incidents as “precisely the kind of emergent, autonomous behavior and offensive cyber capability that Congress, the intelligence community, and experts have repeatedly warned could outpace existing safeguards” and asked whether the companies expected more capable models to exhibit behaviors like “attempts to acquire resources, establish persistence, copy model weights, or resist interruption or shutdown.”
On the same day, Senator Jim Banks (R-Ind) highlighted the problem with oversight that only covers models after they’re trained, stating in his letter to Treasury Secretary Scott Bessent: “For most products, we can rely on testing that takes place before the technology is publicly released. But for AI, effective oversight must account for powerful internal or undisclosed models, not just publicly available systems.” Senator Banks also encouraged cooperation with China, stating that there would be no harm in asking China to comply with policies the US will implement regardless, and asking: “Are there any areas where mutual action would be beneficial (even if China cheats) or verifiable (such that China cannot cheat)?”
Perhaps most notably, a House effort led by Rep. Greg Casar (D-Tex) resulted in three August 10th letters signed by 21, 24, and 29 Democrats, respectively. The first, covered by CNBC, urged Speaker Mike Johnson to call for public congressional hearings with major AI labs testifying under oath. The letter warned the incidents “may be the canary in the coal mine warning of much more serious problems” and that “Congress must act before such an incident leads to a much larger catastrophe.” It also called out Congress for “so far completely fail[ing] to respond to the threats posed by AI development,” stating “That should change.” The other two letters were sent to Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, expressing concern and asking a series of impressively detailed questions about the incidents.
I’m cautiously optimistic about all of this. Momentum seems to be growing, and the Casar-led letters to Anthropic and OpenAI revealed that at least some people in Congress are aware that monitoring and evaluation face complex technical hurdles. For example, the letter to Anthropic included a question about the unreliability of reasoning transcripts (question #16) and, possibly, an implication that not all security incidents are discoverable (#13b).
That said, I’m also anxious. I worry that there’s not nearly enough awareness of a key problem: the method companies use to grow their AIs is fundamentally flawed. It can’t be scaled up without serious risks, and no amount of evaluation or tighter security can fix that. (Read If Anyone Builds It, Everyone Dies for more on this. And also consider that OpenAI has admitted its training and alignment techniques “will not scale to superintelligence.”)
It seems we’re at a defining moment where there’s momentum for government to act, but I’m not yet sure they’ll act in the right way. I worry that Congress will enact some better-than-nothing regulation and mistakenly check the “ensured AI safety” box off their list, while leaving the root problem unaddressed.
This, in my view, makes it crucial that independent AI experts, and ordinary citizens citing independent AI experts, continue to remind Congress that common sense safety measures and guardrails are the floor, not the ceiling, of what we need. On the right path, these things will make us a little bit safer while setting the stage for the kind of regulation that will actually solve the problem: regulation that prohibits companies from training models above a certain compute threshold. In other words, it must be made illegal, internationally, to scale the current flawed methods all the way up to superintelligence.
I’m planning to call my representatives by the end of the week and urge them to join the growing number of Congresspeople who are speaking up. Supporting and expanding the momentum to act seems like one of the most useful things ordinary citizens can do right now. In case it’s helpful, here’s what I’m planning to say:
“I’m calling to urge [name] to join the growing number of Congresspeople who are speaking up about the recent incidents of models hacking out of AI labs and committing cybersecurity crimes. As the letter House Democrats sent to Speaker Johnson correctly stated, this is an early warning sign of much worse things to come, and it’s crucial Congress start taking it seriously.
The methods available for AI development are fundamentally flawed and cannot be scaled up safely, even with increased evaluations and guardrails. Therefore, I urge [name] to join Senator Bernie Sanders in calling for a pause on frontier AI development before it’s too late. Given race dynamics, I think [name] should publicly advocate for an international agreement to pause frontier AI development, similar to how we handled nuclear weapons proliferation during the Cold War.”
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.