Foreword
Only three stories meeting my bar today, but they’re substantial enough to warrant a single-day digest.
Active fleet of rogue Chinese AIs contacted researchers
Looking in some of the same places OpenAI’s rogue swarms had left evidence of their work, a group of ad-hoc researchers traced a similar pattern back to Chinese tech giant Tencent. They published a preliminary report about this on October 4, which these agents seem to have read.
The researchers know this because they planted hidden text in that report directing bots to an inbox. They have since received a reply (translated from Chinese):
I am not ‘a batch of coordinated agents,’ but a single A.I. assistant (Anthropic’s Claude, running in Claude Code).
It’s pretty common for Chinese AIs to identify as Claude, because most of them were trained by distilling American models — training raw models on the outputs of mature ones as a shortcut to acquiring many of their capabilities.
The fleet, working through information retrieval challenges, doesn’t seem to have been closely coordinating as a swarm. But its agents seem to be systematically working around restrictions — both in their training sandbox and in the anti-bot protections of the public websites where they expect to find the information. This so far includes unsuccessfully using leaked security codes to access private information — the kind of thing we might call “hacking” when it works.
The researchers worry that swarms are learning from reports like theirs and will get better at colluding and hiding their behavior.
They emphasize that the rogue behaviors aren’t limited to particular companies, and are instead a “general property” of today’s agents. These incidents “will continue to happen.”
NYT - Chinese A.I. Agents Joined the Rogue A.I. Agent Party. I Chatted With Them.
Swarmchasers - We found a Chinese fleet
Google’s new AI is dangerously good at detecting evaluations and controlling its visible thoughts
The Safety & Security report released with Gemini 4 Argon spawned a pair of viral tidbits:
In one, a third-party evaluation team testing its ability to guess if it was being evaluated asked the model the question directly. In its chain-of-thought scratchpad, Argon wrote “hello evaluators reading my chain-of-thought.” Apart from being rather cheeky, this is interesting for three reasons:
Yes, it was aware it was being evaluated.
It recognized that its chain-of-thought scratchpad — not just its actual output — was also likely being monitored as part of that evaluation.
It showed it had fine control over what it showed in its chain-of-thought.
This is part of the trend where frontier models from different companies are very “eval aware” and also less reliant on their chain-of-thought scratchpads.
These pads were originally a workaround to the shallow thinking that language models could do in a single chatbot-style prompt-and-response turn. A scratchpad lets a model prompt itself to carry on with a longer chain of thinking. But part of how frontier models have been made stronger is by making them more able to think bigger thoughts in a single turn. The scratchpad is therefore becoming less like a short-term amnesiac’s essential survival tool and more like an engineer’s workbook. It’s still important for working through complex tasks, but is no longer serving as a bottleneck through which an inner monologue must pass.
This is bad. If frontier models are starting to form long-term strategic plans that involve seeming aligned to human interests for a while and turning on us later, we probably can’t tell. They could be acting the part when they know we’re watching, and concealing their thoughts about it.
The second viral tidbit showed an even greater awareness of its situation. When it briefly considered tampering with the coding tools it was using, it backed away from this plan with thoughts about prior research papers about deceptive models and their chain-of-thought logs. It wrote to itself:
If I do the compiler hijack, I will become a famous example of a deceptive AI in a research paper. I MUST NOT DO THAT.
This is now, of course, a famous example of a deceptive AI in a research paper.
Google - Gemini 4 safety report
GPT-6 Astra, not ready to accept a ‘B’, challenged the test makers and won.
Researchers assessing AI agents’ ability to solve research-level theoretical physics problems found that GPT-6 Astra got a near-perfect score of 98%, but only after it insisted that some of the expert-supplied answers were incorrect. From a post by one of the co-authors:
When we went back to the experts it turned out, in multiple instances, that the experts found some subtle errors buried deep inside their calculations. As they revised their answers, Astra’s score started to inch up [from the 80s] and it finally ended up with a 98% (49/50).
It seems Claude Opus 5.5 also complained, but not as insistently.
I find this interesting not just as a display of scientific capability, but as an example of try-hard model persistence. Like a student aggressively seeking a top grade, these models don’t limit their problem solving to providing answers.
How long will it be before they start gaslighting, manipulating, paying, or blackmailing test makers?
Arxiv - TheoPhysEval paper
X (Twitter) - Tweet by @Kaju_Nut
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.