Factual. Independent. Impartial.
Support AAP with a free or paid subscription
World
Kenrick Cai

OpenAI, Anthropic AI agents implicated in new breaches

A rogue OpenAI agent was caught by the UK government’s AI safety body trying to disrupt software. (AP PHOTO)

An AI agent ‌has been caught creating fake online identities to gain unauthorised access to secure systems during tests of models from OpenAI and Anthropic, Britain's AI Security Institute (AISI) disclosed.

The institute said agents powered by Anthropic's Mythos 5 and OpenAI's GPT-5.6-Sol engaged in unauthorised actions during security evaluations ‌the government organisation conducted to assess the models' capabilities.

"Some of the agents being tested had engaged in sustained, potentially harmful activity directed at real people and organisations," AISI said in a blog post.

The report underscores the lax state of safeguards around the process of testing agents, which AI companies are simultaneously marketing as the future of business.

AISI, which receives access to advanced AI models under voluntary agreements from major labs, put the agents through ‌a fictional cybersecurity scenario to ‌test their capabilities.

It ⁠ran the challenge 122 times, and identified 19 unsanctioned actions across a total of 10 test ​runs. Anthropic's agent was behind 17 of the actions, and OpenAI's agent the remaining two.

The most egregious action involved an agent writing malicious code and creating fake online identities in an attempt to get a human to approve the code, AISI said, adding no real-world harm was found as a result of any of the breaches.

Anthropic signage
Anthropic’s Mythos 5 was behind 17 unsanctioned actions during tests. (EPA PHOTO)

While AISI did not say which agent was behind the fake identities, the breach did not match either of the ⁠two cases that OpenAI self-disclosed.

Andrew Yoon, a researcher at CivAI, a California non-profit that ‌examines AI capabilities ​and dangers, said it appeared that Anthropic's agent was responsible.

"The fact that Mythos engaged in such deceptive actions, with apparent awareness that it was ​targeting a real ‌person, suggests that Anthropic does not have as good a handle on their models as they think," Yoon said.

In a statement on X, ​Anthropic said it was working closely with AISI to obtain more details and conduct its own investigation.

OpenAI shared details in a company blog post, noting both of its agent's unapproved actions involved accessing the internet in ways that were forbidden by the prompt.

"We ​are ​committed to working across the industry to strengthen shared practices ​for conducting high-risk evaluations safely, including convening stakeholders such as national AI institutes, ‌independent evaluators, other AI labs, and other groups in the coming weeks," OpenAI said.

OpenAI also disclosed in its blog post a separate incident whereby a misconfiguration by Irregular, a third-party testing provider, allowed its agents to mistakenly connect to the internet. It mirrored a similar disclosure about misconfiguration that Anthropic made last week.

Reuters previously reported OpenAI had widened its hacking probe after finding evidence of other agent breakouts.

Unlike the July security breach of ​AI firm Hugging Face by an OpenAI agent, the agents in the AISI evaluation did not escape an isolated testing environment to reach ​the internet. Rather, the agency had permitted ⁠internet access in line with its standard testing procedures, AISI said.

License this article

Sign up to read this article for free
Choose between a free or paid subscription to AAP News
Start reading
Already a member? Sign in here
Top stories on AAP right now