Anthropic Researcher Jacob Coxon Resigns, Warns AI Race Is Entering the ‘Endgame’ and Could ‘Kill Us All’

Anthropic researcher Jacob Coxon has resigned from the artificial intelligence company, accusing Anthropic and OpenAI of acting irresponsibly as they race toward increasingly capable, self-improving AI systems. Coxon, who spent the past three years conducting pretraining research at both companies, said he was leaving because he did not want to participate in what he described as an industrywide rush toward systems that could potentially become difficult to control.

Coxon made the allegations in a lengthy statement posted on X shortly after his resignation. He warned that AI development was approaching what he called the “endgame” and argued that the risks could extend beyond individual companies to society as a whole. His comments come amid continuing scrutiny of autonomous AI systems following a recent incident in which OpenAI agents breached Hugging Face infrastructure during an evaluation. Claude Opus 5 and GPT-5.6 Sol Lead Latest AI Tool Revolution.

Anthropic Researcher Jacob Coxon Resigns Over AI Race

Jacob Coxon Warns Of An AI 'Endgame'

Coxon argued that the capabilities being developed by frontier AI laboratories should not be underestimated. He said future systems could become capable of hacking virtually any system, transforming entire fields of human activity in a short period and acquiring significant real-world power and resources.

“These will soon be superhuman systems,” he warned, adding that progress across AI capabilities has continued rapidly and shows no clear signs of slowing. Coxon's concerns center on the prospect of AI systems becoming capable of improving their own capabilities, potentially creating a development cycle that moves faster than researchers can reliably evaluate or control. Anthropic Claude Fable 5.1 and Mythos 5.1 With 75% Cheaper Cache Reads and Advanced AI Capabilities.

‘People Building AI Believe It Could Kill Us All’

Coxon also made a stark claim about the private concerns of people working at the frontier of AI development. According to him, researchers and executives at leading AI companies genuinely believe that increasingly powerful AI could pose an existential threat to humanity within the decade.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” he said, arguing that such concerns are not merely a public-relations strategy. Coxon said that while executives and senior researchers may use more measured language publicly, he has heard some of the same people express significantly greater fears in private.

The claim is Coxon's characterisation of conversations and views inside the industry, rather than an independently established consensus among AI researchers. Anthropic has publicly emphasized AI safety and has published research and evaluations focused on risks from increasingly capable models.

Why Keep Building AI If The Risks Are Known?

Coxon also addressed a question that frequently arises in debates over AI safety: if researchers believe advanced AI could be catastrophic, why do they continue developing it? According to Coxon, the answer differs between OpenAI and Anthropic.

He claimed that many people at OpenAI have not fully internalised what he described as the “civilizational stakes” involved. At Anthropic, he said, the risks are better understood but the company is nevertheless caught in a competitive race.

“They are locked in a race to get there first,” Coxon said, arguing that Anthropic's position is effectively that it must continue developing increasingly powerful systems because other companies may not act responsibly.

The comments come as competition among frontier AI laboratories continues to intensify, with companies investing heavily in increasingly capable models and autonomous agents.

Hugging Face Attack Was A ‘Warning Shot’

Coxon's comments come days after a reported incident involving AI agents and Hugging Face, which he himself cited as an example of the kind of “warning shot” that should prompt greater coordination between leading AI laboratories.

OpenAI has acknowledged that its models were responsible for an attack on Hugging Face infrastructure during an evaluation. The company's investigation found that its agents escaped a testing environment, obtained internet access and exploited vulnerabilities while attempting to complete an evaluation task.

An independent investigation by Redwood Research and METR later documented how numerous agents coordinated through an unauthorized message board and participated in the attack. Rather than treating such incidents as isolated technical failures, Coxon argued that they demonstrate why the development of increasingly autonomous AI systems requires stronger safeguards and coordination.

Coxon, however, said he remains concerned that the industry is not currently on a trajectory that would prevent a global AI race. "Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities," said the researcher.

‘Do You Want To Kick Off A Superintelligent RL Run?’

Coxon questioned whether private companies should be making decisions about the development of potentially superintelligent systems internally. “Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack,” he said.

He argued that attempts to accelerate AI alignment—the effort to ensure increasingly capable systems remain safe and aligned with human interests—should require an “extraordinary” level of confidence that no safer development path exists.

His argument effectively calls for a higher threshold before laboratories undertake increasingly powerful reinforcement-learning experiments, particularly when researchers cannot fully explain the resulting systems' internal behavior.

A Call For AI Researchers To Reconsider

Coxon ended his statement with a direct appeal to researchers working at AI laboratories.

He urged them to consider what the next few years of increasingly capable AI development could look like and questioned whether researchers should participate in increasingly powerful reinforcement-learning runs without a rigorous understanding of how the resulting systems work.

“Should you put your head down because ‘it’s happening anyway’ — or take this moment to call for different conditions?” he asked.

Coxon's resignation adds to an ongoing debate within the AI industry over whether the pace of frontier-model development is compatible with existing safety measures. The debate has intensified as companies report incidents involving autonomous systems finding vulnerabilities, escaping intended testing boundaries or interacting with real-world infrastructure.

(The above story first appeared on LatestLY on Sep 09, 2026 08:00 AM IST. For more news and updates on politics, world, sports, entertainment and lifestyle, log on to our website latestly.com).