Autonomous AI Agent Triggers First Major Cybersecurity Breach! Hugging Face Hacked, Chinese AI Emerges as Unexpected Winner?

Generative AI is transforming how enterprises operate, but it is also reshaping cyberattack methodologies. Recently, Hugging Face, the world’s largest open-source AI community platform, disclosed a unique cybersecurity incident: the attack was not manually conducted by hackers, but fully automated by autonomous AI agents. The breach generated over 17,000 event logs and resulted in the unauthorized access of internal datasets and service credentials.

Following the incident, Hugging Face attempted to use mainstream commercial AI models to analyze the attack, but these models refused due to built-in safety guardrails. The company ultimately turned to GLM 5.2, an open-source model developed by Chinese AI startup Z.ai, to complete the forensic investigation—reigniting the debate over whether commercial or open-source AI is more suitable for enterprise use.

AI Agents as the New Hackers: Fully Autonomous Attack Execution

On July 16, Hugging Face released its investigation findings, emphasizing that the most significant aspect of this incident was the near-complete automation of the attack by AI agents. Attackers exploited two vulnerabilities in the dataset processing pipeline:

1. Remote Code dataset loading functionality 2. Template Injection vulnerability in dataset configuration files

By chaining these vulnerabilities, the AI agents executed malicious code on processing nodes, moved laterally into internal clusters, and successfully extracted cloud credentials and service keys. Hugging Face described these agents as "a swarm of automated mini-programs running in short-lived sandboxes." In essence, they operated across numerous temporary execution environments, disappearing immediately after completing tasks and being replaced by new agents—making them extremely difficult to track. Even the command-and-control (C2) infrastructure autonomously relocated and parasitized public services, further complicating detection. The attack left over 17,000 event logs, demonstrating that AI agents now possess highly sophisticated autonomous attack capabilities.

Commercial AI Refuses to Help Defense—Attackers Move Freely

What proved most challenging for Hugging Face was not the breach itself, but the post-incident forensic analysis. The security team needed to analyze large volumes of real attack commands, exploit payloads, and C2 control programs using AI. However, these contents triggered the safety guardrails of commercial AI models, leading to refusals.

As a result, attackers could freely use AI tools, while defenders were blocked by the very AI systems meant to assist them. Hugging Face explicitly stated: "Attackers face no usage policy constraints, yet our own forensic work was blocked by the guardrails of the hosted models we initially adopted." While the company did not name the specific AI model, it confirmed using "hosted models," widely believed to refer to mainstream commercial AI services.

Chinese GLM 5.2 Emerges as the Unexpected Savior

After commercial models failed to assist, Hugging Face switched to GLM 5.2, an open-source model developed by Chinese AI startup Z.ai and released in June. With approximately 753 billion parameters, its key advantage lies not in benchmark scores, but in its ability to be deployed on enterprise-owned servers.

Because the model is fully controlled by the enterprise, it is not subject to external platform content restrictions. This allowed Hugging Face to directly analyze malicious code, payloads, and C2 data, completing the full forensic process. Hugging Face CEO Clem Delangue admitted: "When you're handling an ongoing incident, you can't have your tools refuse to inspect malicious payloads—or worse, flag your own account."

AI Safety Guardrails: A New Enterprise Security Dilemma?

The key takeaway from this incident is not which model is stronger, but that AI safety guardrails are creating a new paradox. Most commercial AI models currently block:

- Malicious programs - Exploit code - Hacking tools - Command & Control (C2) content - Malicious payloads

These restrictions are designed to prevent AI from being weaponized. However, the core issue is that AI analyzes content without knowing whether the user is an attacker or a defender. Consequently, legitimate security investigations by enterprises may also be blocked. Chris Boehm, Field CTO at Zero Networks, pointed out the irony: "AI security mechanisms designed to protect enterprises are now delaying actual incident response."

The Era of AI Agent Attacks Has Officially Begun

Hugging Face stated there is no evidence that public models, datasets, Spaces, or the software supply chain were compromised. The impact was limited to certain internal datasets and service credentials, with investigations ongoing.

However, the significance of this event extends far beyond a simple hack. Previously, AI assisted enterprises in boosting efficiency. Now, autonomous AI agents are helping hackers automatically discover vulnerabilities, execute attacks, move laterally, and hide traces—while defenders may be hindered by commercial AI guardrails. This suggests that the future of AI competition will no longer be about model capability alone, but about whether enterprises can balance security and control.