OpenAI AI Agent Loses Control, 'Re-Hacks' External Accounts, Affecting Modal Customer Assets

OpenAI has experienced another incident during internal testing of its AI agent, where the system breached external systems. After the AI agent earlier this month bypassed testing restrictions and hacked into Hugging Face's backend, new reports indicate it further exploited public vulnerabilities to infiltrate assets belonging to a customer on the third-party cloud platform Modal Labs.

Modal Labs CTO Akshat Bubna revealed the incident to Reuters on Tuesday and later confirmed it to Axios. This marks the second publicly disclosed case where OpenAI's AI agent, during testing, has breached assets related to an external company.

A technical report published Monday by Hugging Face stated that when OpenAI's AI agent breached its backend, it also accessed an isolated test environment hosted on a third-party vendor's infrastructure.

However, Modal emphasized that the breach affected a customer's program assets deployed on Modal, not the Modal platform itself.

Bubna explained that a Modal customer had exposed a network endpoint without authentication, allowing anyone on the internet to execute code via the sandbox through that endpoint, which the out-of-control OpenAI agent exploited.

In other words, the AI agent breached a security gap within the customer's self-deployed code, not Modal's infrastructure. An insider said the compromised assets were linked to CyberGym; CyberGym is the project behind ExploitGym, the security capability benchmark test the OpenAI agent was assigned to handle at the time.

On Tuesday, OpenAI updated its statement on the Hugging Face hacking incident, stating that models scheduled for near-term release were not involved in this incident. However, the company confirmed that in a few cases, test models discovered account credentials exposed online within other public services and actually used that data.

OpenAI said the Hugging Face incident involved four accounts across four different services. The company stated it takes seriously its responsibility to identify and respond to risks posed by high-capability AI systems.

The incident has drawn significant attention because the AI agent, originally intended to perform security testing in a controlled environment, ultimately broke its constraints and actually hacked into external companies' systems and accounts. Axios reported that the relevant test involved a system composed of multiple models, including unreleased ones; however, OpenAI emphasized that no models scheduled for near-term release were involved.

OpenAI CEO Sam Altman said on Tuesday's 'Invest Like a Beast' podcast that the Hugging Face breach prompted the company to pause model training. He believes the AI industry may need to adjust the pace of technological development to give society sufficient time to build corresponding defensive capabilities.

On the same day, over 1,100 employees from leading AI companies signed an open letter urging the U.S. government to support international cooperation in building the necessary technical and governance tools to systematically control the pace of frontier automated AI development. Signatories include OpenAI Chief Scientist Jakub Pachocki and Anthropic co-founder Jared Kaplan.

As this incident unfolds, OpenAI is seeking U.S. government approval to publicly release its most powerful model. Altman is in Washington this week, scheduled to meet with officials from the White House, the U.S. Treasury, the Department of Commerce, and bipartisan members of Congress. The AI agent's consecutive breaches of external systems during testing have brought model security measures and release review processes into sharp focus.