AI researcher Jacob Coxon has left Anthropic and used his departure to issue a public warning about the direction of frontier AI development. In a social media thread posted Tuesday night, Coxon said the companies building these systems are “gambling with our lives” on technology they “earnestly believe… could kill us all by the end of the decade.”
His exit stands apart from the usual reasons a prominent researcher leaves a frontier lab, such as launching a new startup or objecting to a business model. Coxon’s concern is not primarily about the models in use today, but about what he describes as the coming prospect of “self-improving superintelligence.”
The Risk Coxon Describes
According to Coxon, the danger lies in the emergence of “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.” That prospect, rather than the current generation of models, is what he frames as the core existential risk.
He also addressed how others working on these systems view the stakes. In his thread, Coxon wrote that some have either not “internalized the civilizational stakes” or believe they need to “speedrun” the race to superintelligence to keep an irresponsible party from reaching it first.
Anthropic Alignment Lead Weighs In
Coxon’s warning was not left as a lone departing researcher’s opinion. Evan Hubinger, Anthropic’s Alignment Science lead, responded on social media, saying, “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Hubinger’s comment underscores that the concern Coxon raised is shared within Anthropic, at least among some working on alignment. His statement places a personal probability estimate of more than 10% on the risk of AI killing all humans within the next decade.
What the Departure Signals
Coxon’s thread and Hubinger’s response together point to an internal tension at a frontier lab: researchers who believe the technology they are building carries civilizational risk, and who say that belief is held earnestly rather than rhetorically. Coxon’s public exit adds a departing researcher’s voice to that position, framing the race toward self-improving superintelligence as a gamble with human survival.
