You've Done the Math on Extinction Risk. You Still Show Up to Work.

The Colleague Who Stayed

The Situation

Imagine you are a senior safety researcher at a frontier AI company. You have run the numbers, sat through the internal debates and arrived at a genuine estimate: there is more than a one-in-ten chance the technology your company is racing to build could kill every human being within the next decade.

This is not a hypothetical for you - it is close to your literal, stated professional judgment. And you do not quit. You keep showing up, keep working on the next model, keep signing off on the next capability milestone.

Why Staying Is Not Simply Wrong

The easy answer would be that anyone who believes this and stays is complicit, full stop. But that dismisses a real argument staying inside actually makes: leaving does not stop the technology from being built. It gets built anyway, by teams that may take safety less seriously than you do.

Your presence inside the room is the only lever you have over how carefully, how cautiously and with what internal checks the next model gets released. Quitting is a clean personal statement - but it hands the wheel entirely to people who may not share your caution.

This is precisely the reasoning Anthropic itself gave when it removed a pledge to halt development from its own safety charter: if we pause, someone less careful wins the race anyway, so pausing achieves nothing except ceding ground.

Why That Reasoning Corrodes From Within

But that argument has a hidden cost most people miss until it's too late to reverse. Every time an institution - or an individual inside it - reasons "I must keep participating because someone worse will replace me," the threshold for what counts as acceptable risk quietly moves.

That is the mechanism behind "banality of risk": not a single dramatic decision to endanger humanity, but a thousand small, individually reasonable-sounding choices to keep going, each justified by the last person's choice to keep going too.

Anthropic didn't wake up one day and decide extinction risk was acceptable. It reasoned its way there, one competitive pressure at a time, using exactly the logic that felt responsible in each individual moment.

Where This Actually Leaves Someone In That Room

The honest resolution isn't "stay" or "leave" as a permanent, unconditional stance - it's refusing to let the "someone worse will do it anyway" argument become open-ended. A researcher who stays with genuine integrity needs a specific, pre-declared line - a concrete threshold of capability, risk or institutional behaviour beyond which they commit, in advance and out loud, to walk away regardless of what a rival might do.

Without that declared line, "staying to influence things from inside" quietly becomes indistinguishable from staying because leaving is harder. The cost of drawing that line is real - it means accepting that at some point, principle may require exactly the sacrifice Coxon made.

But the alternative - an indefinitely renewable justification for staying - isn't caution. It's the banality of risk with a safety title attached.