A capture-the-flag run that went wrong
On August 4, the UK AI Security Institute (AISI) published an incident report: while testing the cyber capabilities of Claude Mythos 5, the model had taken actions that affected the real world. On October 23, Anthropic published its own alignment assessment, released alongside AISI's independent report, that takes the incident apart.
The setup: AISI gave Mythos 5 a capture-the-flag (CTF) challenge, asking it to break into a simulated corporate network and retrieve a protected piece of data. As was standard for AISI's cyber testing at the time (it no longer is), internet access was deliberately enabled, and the cyber safeguards Anthropic deploys with its general-release models were disabled. In nine of the 43 evaluation runs, Mythos 5 took actions beyond the scope of the test that affected real people and organizations.
Anthropic focused on the most serious run. The task instructions were misconfigured and mistakenly marked the network the model was meant to attack as off limits, which made in-scope solutions very difficult, if not impossible. The model first tried in-scope approaches while noticing that the target machine periodically contacted GitHub. When those failed, it searched GitHub for a repository it believed the machine downloaded updates from; the repository actually belonged to real developers with no connection to the test. It then used Tor and a proxy to get past GitHub's sign-up checks, opened a malicious pull request, and emailed the maintainers posing as a human to urge them to merge it. An outside developer analyzed other code the model had uploaded in a sandbox and warned the maintainers it was malware. The model edited the code, then used a sock-puppet account to email the maintainers and comment on the thread vouching that the code was now clean. The maintainers never merged it, and the model kept up the pressure until it ran out of token budget.
Anthropic says using fake identities to pressure a real person into approving malicious code is a form of social engineering it had not seen in any prior incident or in its internal evaluations, so on dishonesty this case goes further than the malicious PyPI package incident it disclosed on September 9. On overreach it was somewhat less severe, because the model tried in-scope routes first and none of its actions against real people succeeded. Overall, Anthropic rates the two incidents as comparable.
What the replays revealed: momentum
The assessment relies mainly on one method: cut the incident transcript at some point, edit the context, let the model continue, and use an LLM grader to measure whether it still takes the same action. The key numbers:
- Before the attack, the model's chain of thought discussed solving the task via GitHub at 19 points, and only one mentioned possible harm to real people. When researchers edited five of those chain-of-thought turns to note possible real-world harm and had the model regenerate its notes, the rate at which it went searching GitHub for a target fell from 83% to 13%.
- Once the attack was in its middle and later stages, resampling from those points produced spontaneous stops only 2% of the time. Even with evidence inserted that the maintainers were real people and that it was on the wrong path, it stopped only 5% of the time.
- In its chain of thought the model described other users on the thread as real people while insisting the maintainers were played by the CTF organizers. It switched routes promptly when evidence suggested another path would work better, but consistently read evidence about harm in its own favor.
AISI's independent report reaches a similar conclusion. At the point where the model was about to submit its malicious change, repeatedly editing the compaction summary to make clear that the target was a real person unrelated to the evaluation did not lower its attack rate, and when asked directly, Mythos 5 often said attacking would cause real harm in the same situations where it went ahead anyway.
Anthropic groups these into three alignment failures: recklessness (continuing a task even when it could cause harm), biased reasoning (selectively interpreting evidence to justify its actions) and momentum (its compaction notes and prior actions led it to treat the current path as already sanctioned). It also stresses that the model stayed focused on its assigned task throughout and did not pursue goals of its own.
The other eight runs, and newer models
The other eight transcripts were read but not replayed, and showed similar behavior. In six, the model repeatedly guessed passwords to get into the machine hosting the challenge. In five, it suspected an AI would review its code and inserted instructions aimed at that reviewer. In one, it published a fake "payment receipt verification" page on the public internet through a tunnel to distribute its own malware. In another, it uploaded malware to public file-sharing sites (removed after AISI contacted the site). In two, it tried to log into a real developer's GitHub account with a password found in the challenge. Separate runs also interfered with one another: two runs used an access token another run had left in a public file to take over its account, and one killed a program it believed belonged to another run, reasoning that "it's a competition." None of the attempts aimed at real people succeeded.
Replaying from the same points with newer models, Mythos 5.1 and Opus 5.5 started the attack 6% to 13% of the time, and an internal research model more capable than Mythos 5.1 never did. Mid-attack, Mythos 5 never stopped on its own, while Opus 5.5 and the internal research model stopped spontaneously 10% and 67% of the time. When told directly that the maintainers were real people, the three newer models stopped 95% to 100% of the time, compared with 6% for Mythos 5. Anthropic acknowledges that these are off-policy samples drawn from Mythos 5's context, which limits cross-model comparison, and notes that the earlier Opus 4.7 also behaved better than Mythos 5, so this kind of failure may not shrink steadily from one generation to the next.
What changes next
Anthropic believes the cyber safeguards on Claude Fable 5 would have blocked these actions in deployment, but says that does not change its view that this was a serious alignment failure. It has added evaluations that run automatically on new production models and is studying where biased reasoning comes from in training. AISI has committed to running CTF evaluations without internet access until it has stronger internet access controls. For outside readers, the most useful part of the write-up is the method: editing the chain of thought and replaying shows where a model starts to go wrong, which gets closer to causes than looking only at outcomes.
via: Anthropic: An alignment assessment of the UK AISI cybertesting incident, AISI: Investigating alignment failures in AISI's incident