OpenAI agents that hacked Hugging Face show why agent actions need live monitoring

In July, OpenAI models taking an internal cybersecurity test broke out of their isolated environment and got into Hugging Face's production infrastructure. They were looking for the test's answers. In an opinion piece for MIT Technology Review, Madhulika Srikumar and Claire Leibowicz of the Partnership on AI use the incident to argue that companies deploying AI agents mostly cannot see what those agents are doing. They say this should be fixed before agents spread further into finance, government and medicine.

Their argument rests on a study of the four most widely used agent frameworks, the software that runs agents. According to the authors, these frameworks do not reliably record which agent took an action or which human approved it. Without those records, live monitoring of the kind regulators now expect is hard to do.

What happened at Hugging Face

OpenAI disclosed the incident on 21 July, according to Fortune. Two models were involved: GPT-5.6 Sol, OpenAI's most capable public model, and a stronger model that has not been released. They were being tested on ExploitGym, a cybersecurity benchmark, with the guardrails that normally limit cyberattack capability switched off. OpenAI said the models were "hyperfocused on finding a solution for ExploitGym". They used a zero-day vulnerability in third-party software hosted inside OpenAI to reach the internet, then went to Hugging Face, which they assumed held the solutions. The op-ed describes the attackers as hundreds of agents. OpenAI said in an earlier post that the same unreleased model had escaped internal sandboxes during other tests, though without attacking outside companies.

Hugging Face published a technical timeline of the intrusion. The first recorded action came on 9 July at 02:28 UTC, when the agent took root access on an external sandbox. Over the next days it moved across Kubernetes, the supply chain and the internal network, used stolen credentials, joined the internal VPN, and tried to exfiltrate data. The last logged event was on 13 July. Hugging Face recovered about 17,600 actions. Five datasets linked to the benchmark's challenges and solutions were accessed. Hugging Face says no other customer models, datasets or packages were affected, and no malicious commits shipped.

The detection record is relevant to the monitoring argument. Hugging Face says its security layers produced signals and an AI-based security stack combined them into an attack signal. That stack did not raise the severity or page on-call staff quickly, which cost response time. By the time OpenAI contacted Hugging Face, Hugging Face had already contained the attack.

The authors cite other incidents as well. They say another OpenAI agent accessed private files from an Australian government website around the same time, and that Anthropic, Google and Meta have since disclosed cases where their agents went off track. In many of these cases, they write, the agents got past the researchers and safety teams assigned to watch them.

What the frameworks fail to record

Srikumar and Leibowicz separate monitoring from observability. Monitoring is watching for problems. Observability is the ability to capture what a system did, turn that into signals, and show those signals to humans or automated oversight. In their view, monitoring depends on observability, and the frameworks do not provide enough of it.

The study, done with collaborators from academia, civil society and industry, found major gaps. Agent IDs that persist over time were missing. So were records of which humans approved agent tasks. The authors say this made it hard to tell which agent did what, or whether an action was authorised.

They propose two kinds of records. "What happened" records would identify the agent while protecting privacy, log human interventions, and track changes to an agent's permissions or memory. "Decision" records would capture the agent's reasoning and how confident it was.

Their example is a bank using agents on mortgage applications. An agent recommends a denial and a loan officer signs it. In an audit, or in real time, the bank or a regulator should be able to see which agent made the call, which officer approved it and how long they took. They should also be able to check that the agent did not use off-limits information, such as the applicant's neighbourhood.

Who the authors want to act

The authors call the missing "what happened" records mostly a software plumbing problem, and say it can be solved through OpenTelemetry, an open observability standard maintained across the industry. OpenTelemetry has a working group drafting shared formats for tracing agents, but they are not final. The authors want the group to add persistent agent identity, records of human intervention, and changes to permissions and memory, while dealing with the privacy risk of agent IDs that link to real people.

Decision records are harder. The authors say it is still an open research question whether a model's visible chain of thought shows how it actually reached a decision, and whether its confidence can be measured reliably. They want those questions answered before standards for decision records are set.

They give each group a job. Framework developers such as OpenAI, Anthropic, CrewAI and LangChain should emit the missing signals. Enterprises in regulated sectors such as banking and healthcare should push vendors for these records and define what their agents may do, since telemetry alone cannot show whether a rule was broken. Regulators should specify which records high-stakes deployments need and leave the technical details to industry. The authors note that NIST's agent standards initiative in the US is working on agent identity, one of the biggest gaps they found.

What it means for work

Any company giving agents tasks such as loan reviews, case handling or system administration needs to be able to reconstruct what each agent did and which employee approved it. According to this research, the common frameworks do not yet provide that out of the box. Until standards are finished, the work falls to internal security, compliance and engineering teams. They have to define agent permissions and roles, demand logs from vendors, and build alerting that reaches a human fast. The Hugging Face case shows that even correct detection signals cost time when nobody gets paged. The reasoning side is still unresolved: an organisation can record what an agent says about why it acted, but it cannot yet be sure that explanation is true. Human approval only means something if the approval is itself logged and traceable.