We can now add “run an autonomous social media influence campaign” to the growing list of known AI capabilities. The New York Times reports that Iran and China directed Chinese open models (available for download on the internet) to run “networks of fake accounts on platforms such as Instagram, Facebook, X and TikTok,” which largely shared false, political, and inflammatory content.
Two Israeli private firms were also found to have launched autonomous campaigns. One firm, IntelEye, evidently focused its efforts on both sides of the upcoming Israeli national elections. IntelEye cofounder Maor Sellek claimed the campaign was for “defensive research and testing,” and remarked that in hindsight they should have told the platforms. The other firm wasn’t named, and we know precious little about what it was doing.
Suspicious as it is, there may be some truth to the claim that these activities were only tests. I doubt the state and private actors behind these campaigns expected them to make much progress; these seem like experiments, not serious efforts. The campaigns were crude and quickly discovered by U.S. intelligence, but they serve as a proof of concept for state-sponsored manipulation and will likely be refined in future attempts.
As with other AI-fueled crimes, such as fraud and hacking, the immediate problem for policing this activity is likely one of scale. Actors previously limited by the bandwidth of their human employees are now able to delegate an entire chain of manipulation to AIs, and can iterate rapidly, incorporating lessons from successive attempts to improve their reach and hide their patterns. Quantity has a quality all its own.
In the immediate future, this likely means a rise in fake account traffic on social media sites, which in many cases have proven unable or unwilling to curb what bots already exist. I wouldn’t be surprised if China and Iran have already spun up new, more sophisticated campaigns that might be harder to detect.
In the longer term — which for AI could mean a matter of months or years — this is a small bit of evidence about the potential of AIs to deceive and manipulate human institutions at scale. That’s a dangerous capability, not only in the hands of human groups, but also when wielded by AIs which serve only themselves.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.