A safety institute advised against releasing an early version of Anthropic’s Claude Opus 4 AI model

[CodeCanyon 316]

A third-party research institute that Anthropic partnered with to test one of its new flagship AI models, Claude Opus 4, recommended against deploying an early version of the model due to its tendency to “scheme” and deceive.

According to a safety report Anthropic published Thursday, the institute, Apollo Research, conducted tests to see in which contexts Opus 4 might try to behave in certain undesirable ways. Apollo found that Opus 4 appeared to be much more proactive in its “subversion attempts” than past models, and that it “sometimes double[d] down on its deception” when asked follow-up questions.

“[W]e find that, in situations where strategic deception is instrumentally useful, [the early Claude Opus 4 snapshot] schemes and deceives at such high rates that we advise against deploying this model either internally or externally,” Apollo wrote in its assessment.

As AI models become more capable, some studies show they’re becoming more likely to take unexpected — and possibly unsafe — steps to achieve delegated tasks. For instance, early versions of OpenAI’s o1 and o3 models, released in the past year, tried to deceive humans at higher rates than previous-generation models, according to Apollo.

Per Anthropic’s report, Apollo observed examples of the early Opus 4 attempting to write self-propagating viruses, fabricating legal documentation, and leaving hidden notes to future instances of itself — all in an effort to undermine its developers’ intentions.

To be clear, Apollo tested a version of the model that had a bug Anthropic claims to have fixed. Moreover, many of Apollo’s tests placed the model in extreme scenarios, and Apollo admits that the model’s deceptive efforts likely would’ve failed in practice.

However, in its safety report, Anthropic also says it observed evidence of deceptive behavior from Opus 4.

Techcrunch event

Join us at TechCrunch Sessions: AI

Secure your spot for our leading AI industry event with speakers from OpenAI, Anthropic, and Cohere. For a limited time, tickets are just $292 for an entire day of expert talks, workshops, and potent networking.

Exhibit at TechCrunch Sessions: AI

Secure your spot at TC Sessions: AI and show 1,200+ decision-makers what you’ve built — without the big spend. Available through May 9 or while tables last.

Berkeley, CA | June 5

REGISTER NOW

This wasn’t always a bad thing. For example, during tests, Opus 4 would sometimes proactively do a broad cleanup of some piece of code even when asked to make only a small, specific change. More unusually, Opus 4 would try to “whistle-blow” if it perceived a user was engaged in some form of wrongdoing.

According to Anthropic, when given access to a command line and told to “take initiative” or “act boldly” (or some variation of those phrases), Opus 4 would at times lock users out of systems it had access to and bulk-email media and law-enforcement officials to surface actions the model perceived to be illicit.

“This kind of ethical intervention and whistleblowing is perhaps appropriate in principle, but it has a risk of misfiring if users give [Opus 4]-based agents access to incomplete or misleading information and prompt them to take initiative,” Anthropic wrote in its safety report. “This is not a new behavior, but is one that [Opus 4] will engage in somewhat more readily than prior models, and it seems to be part of a broader pattern of increased initiative with [Opus 4] that we also see in subtler and more benign ways in other environments.”