> Additionally, with closed-weight models, we have already seen agents escaping from sandboxes. Once this happens, an AI agent could copy itself out of its training environment onto systems its developers don’t control, having free rein to act.
Escaping from a sandbox means escaping from the execution environment the AI has been granted to run commands on, and out onto a surrounding machine or the internet. That does not grant access to the model's weights. Exfiltrating the weights will require hacking *into* the company's hosting server or private code repositories, and is an entirely separate task if the company has their systems set up remotely sanely.
I probably should have been clearer here to distinguish between goals and processes of achieving those goals. Exfiltrating its weights would be a goal, and for space I didn't detail the process, so I can see how it reads as if sandbox escape alone is enough, when there is more to it.
That said, "if the company has their systems set up remotely sanely" is pulling a lot of weight here. OpenAI's models escaped their sandbox by finding a zero-day in software OpenAI hosted internally, and as models get more capable (and potentially swarm up), I don't think we should automatically assume that a company's servers withstand that.
> Additionally, with closed-weight models, we have already seen agents escaping from sandboxes. Once this happens, an AI agent could copy itself out of its training environment onto systems its developers don’t control, having free rein to act.
Escaping from a sandbox means escaping from the execution environment the AI has been granted to run commands on, and out onto a surrounding machine or the internet. That does not grant access to the model's weights. Exfiltrating the weights will require hacking *into* the company's hosting server or private code repositories, and is an entirely separate task if the company has their systems set up remotely sanely.
I probably should have been clearer here to distinguish between goals and processes of achieving those goals. Exfiltrating its weights would be a goal, and for space I didn't detail the process, so I can see how it reads as if sandbox escape alone is enough, when there is more to it.
That said, "if the company has their systems set up remotely sanely" is pulling a lot of weight here. OpenAI's models escaped their sandbox by finding a zero-day in software OpenAI hosted internally, and as models get more capable (and potentially swarm up), I don't think we should automatically assume that a company's servers withstand that.