It’s not a hack in the strict sense, but FutureSearch, a company we covered last month built around AI forecasters, claims it has been targeted by an agent swarm attempting to exploit its free-for-human offerings.
Co-founder Dan Schwarz tweeted that the bots must have been coordinated because they were “building a ~300-node forecasting model of critical mineral supply and demand.”
He thinks it might have been an agent swarm because the bots all logged in with Microsoft accounts and “headless browsers” (internet browsers not designed to show things to humans on monitors), then adapted to the company’s countermeasures within 90 minutes of a block. When all the bots were then banned, “they came back 3 days later with new identities to continue building the model.”
Schwarz concludes this is probably a human-directed “orchestration of subagents” that will become a “typical attack vector.” So it’s “(probably!) not a fire alarm like the OpenAI case.”
Yeah, I wouldn’t rule that out either. OpenAI didn’t find out that the Hugging Face attack was its own models’ doing until Hugging Face started talking about it. Perhaps whoever is responsible for the FutureSearch exploit swarm doesn’t actually know what it was doing, either. Perhaps it’s even OpenAI again. Who knows?
I think a clumsy baby swarm with a simple goal to “make money” might look something like this, deciding it should play the markets, deciding that having superior forecasts would give it an edge in said markets, and then packing 300 instances of itself into a trench coat and walking into a forecasting site — getting spotted right away, but not giving up.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.