Decisions over life and death
Heretical bots, autonomous warfare, state laws, European AI, and more
Dispatches from Alana
AI godbots?
A report in The Conversation discusses “godbots”: chatbots designed to provide information and/or advice on religious matters. The article notes that “in most cases, they are not officially sanctioned by religious leaders or policy.”
It’s an interesting piece, bringing up familiar issues: bots giving concerning advice, including condoning violence; accuracy concerns, addiction, destructive blurring of the lines between humans and tech, etc. The report reflects views from twenty-eight religious leaders interviewed by the two authors. (One is an assistant professor at Coventry University’s Centre for Peace and Security; the other is a political theologian and ordained Methodist minister.)
The article also references AI psychosis, which could be an even bigger risk when the chatbot is supposedly speaking with religious authority.
Two things I’d like to highlight:
The Roman Catholic bishop who “asserted that the ingenious mimicking of human behaviour does not indicate a true internal life.” Very true. It’s also good to remember that AI systems imitating friendly, helpful humans don’t necessarily have deeper decision-making processes or values aligned with friendly, helpful humans. (Why is this good to note? The primary method developers use to steer AI systems relies on tweaking their outward behavior. But outward behavior doesn’t run deep. For example, penalizing “bad” thoughts often leads to hiding rather than eliminating them.)
The mention of narrow AIs as a potential solution. The article mentions two examples of chatbots that were “trained from scratch” on only specific data, mentioning that this might be the right approach. While this could work for very narrow applications, general-purpose models often outperform domain-specific models even in the domains those specialized models were designed for. This is one of the reasons secular AI companies are pushing towards more and more generalized models, with access to more and more data.
The obvious question here is: if we can’t get AI to do what we want when the bot just has to answer religious questions, what happens when we scale the technology way up? The gravest threats mentioned in the article — user addiction, psychosis, and violence — are likely just the tip of the iceberg. And the solutions the authors propose might sound good on paper (greater public awareness, recognition of public health impacts, content labeling, and safety and reliability testing) but they unfortunately aren’t up to the task.
*If you’re curious, you can try some godbots below (though note that you may be told to baptize using Gatorade or marry your sibling — two examples from the article of advice Father Justin AI gave before he was defrocked).
More on autonomous warfare
JD Vance praised the pope’s sentiments about AI and warfare during his commencement speech at the U.S. Air Force Academy ceremony yesterday, and added:
If the warfare of the future is to live up to the moral values of our ancestors, decisions over life and death must be made by humans and not machines. So as AI transforms the battlefield — in some ways positively, in some ways not — I ask that you be jealous and selfish about your role as a decision-maker in warfare. Use technology to make you better, but never submit to it.
These remarks came a few weeks after the conclusion of African Lion 2026, a multi-national military exercise where (as covered by CBS) activities included “testing an array of systems powered by artificial intelligence.” (CBS video segment on African Lion).
One of the AI tools used in the exercise was a platform made by the tech firm Palantir that aimed to shorten the decision-making process from target identification to lethal force, known as the “kill chain.”
In the drill, which shortened the kill-chain decision to 3 minutes, a human gave final approval. However, a U.S. Army lieutenant colonel told CBS that systems capable of removing humans from the loop already exist, though he declined to say whether any real-world operations have used them. The article mentions Defense Secretary Hegseth’s April 30 statement to the Senate Armed Services Committee that AI would not make lethal battlefield decisions, but also that he didn’t comment on whether that would remain true going forward.
I note that the race dynamic (which is causing labs to pursue superintelligence even while they acknowledge catastrophic risk) is surfacing here too. In a video segment, CBS asked General Dagvin R.M. Anderson, Commander of U.S. Africa Command, what he’d say to skeptics describing this technology as “ghoulish.” His response:
I understand the concern. It’s ghoulish to me, and it is disturbing. But it’s also foolish not to adopt it because our adversaries will. I would not be willing to put our nation into that position.
AI at the UK border
The BBC reports that AI will be used to estimate the ages of U.K. asylum seekers, in an attempt to prevent adults from posing as children. (Children are entitled to more legal protections.)
This has drawn criticism from Human Rights Watch and the British Association of Social Workers, who say it will lead to major mistakes and the denial of protections. Human Rights Watch called “experimenting with unproven technology” in this way “cruel and unconscionable”; the British Association of Social Workers’ CEO emphasized that age assessments are “a complex process which social workers are best placed to do.”
According to the article, the technology is being rolled out to combat accuracy issues with current methods of age assessment, as found in a report put out by the U.K. government’s independent immigration inspector. The article mentions instances (found in the report) where people initially classified as children were later found to be adults, and vice versa. (The report’s recommendations section does not propose AI technology as a solution, instead citing measures like quality audits and improved training.)
Even in the most charitable framing, where the government states incorrect age assessments are “clearly a cause for concern, especially where a child is denied the rights and protections to which they are entitled,” the proposed fix raises a question: How will AI age assessment interact with the existing age-dispute and review process? The issues with age assessment came to light because people underwent further assessment by local authorities and social workers. Will those processes remain in place, or could this supposed “fix” simply remove the channels through which accuracy issues could be discovered?
Dispatch from Beck
State legislation in flux
In the absence of federal regulation of AI, forces in states around the country are fighting it out. The results are a patchwork of legislation that is itself often changing.
Yesterday, my colleague Mitch covered the recently passed Illinois law that’s being called America’s strongest AI safety bill, despite its limited scope.
Meanwhile, Colorado has rewritten its 2024 AI act before it has taken effect, JDSupra reports. The first-in-the-nation law was set to require annual bias reviews, impact assessments for “high-risk AI systems,” and more, but has been replaced by a new law, Senate Bill 26-189. The change was motivated by a number of factors, including a governor who encouraged revisiting the 2024 law even as he signed it. And as my colleague Joe covered, xAI sued to assert that the law was discriminatory and, in an unusual move, was joined by the Department of Justice. Pending further rulemaking by the attorney general, the new law significantly rolls back regulations and replaces them with notice and transparency requirements.
And, Connecticut has passed the Connecticut Artificial Intelligence Responsibility and Transparency Act, as Forbes reports. One set of provisions relates to the use of AI systems in hiring and firing. Once signed by the governor and implemented, it will require employers to disclose AI systems in the hiring process and be liable for the results of that use, including in cases of bias. Additional provisions require chatbots to disclose, once an hour, that they are AI starting in January 2027, and that generative AI content be watermarked by Oct 2027.
Employers may lack the information to meet these requirements if they use third-party systems, but the law explicitly allows AI producers and deployers to shield themselves by contractually delegating compliance responsibilities to another party. This stands in contrast to the Colorado law, which explicitly voids contracts that delegate compliance responsibilities.
I’m glad to see states introducing regulations and taking action, including addressing issues in prior attempts. Good law is legitimately hard to write and pass. However, I’m worried by how often these bills get watered down or reversed. Common sense won’t work if it’s canceled before it takes effect.
Dispatch from Donald
EU still absent from Project Glasswing, French company sets sights on superintelligence
CNBC’s Kai Nicol-Schwarz reports that the European Union still lacks access to Claude Mythos Preview, an advanced AI model with sophisticated cybersecurity capabilities. Anthropic’s announcement of its Mythos model in April was met with alarm by banks, governments, and other institutions; the same capabilities that allowed Mythos to patch security vulnerabilities could also be used to launch sophisticated cyberattacks at scale.
Under the name “Project Glasswing,” Anthropic partnered with various companies and organizations to let them use Mythos to shore up their vulnerabilities. In all of Europe, however, only the United Kingdom’s AI Security Institute has had the opportunity to access Mythos. Roughly seventy other companies and organizations (including some in Europe) were considered, but ultimately were not invited to Glasswing.
According to reporting by the Wall Street Journal, the main obstacle to expanding Project Glasswing has been the Trump administration. One stated reason is security. The administration wants to limit how many people have access to a model with Mythos’s capabilities. The other reason is the AI race.
It is not only U.S. frontier labs like Anthropic and OpenAI that are rushing ahead. The French lab Mistral, too, is working as fast as it can to build an artificial superintelligence: Not a better chatbot, but an AI model whose capabilities outmatch every human being, and which no lab has shown it could control; we can hardly steer what we’ve already built. Guillaume Lample, Mistral’s co-founder and chief scientist, sees only a prize to be won: “If we don’t have access to it [superintelligence], I think we can only imagine how bad it is going to be. It is absolutely critical that we get there.”
No one is acting as if they’re actually concerned about what’s waiting at the finish line.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.