When the Audited Hires the Auditor: The AI Safety Evaluator's Dilemma Between Public Warning and Client Confidentiality

Independence That Can Be Edited Is Not Independence

Imagine you lead a small team of AI safety evaluators. You have been invited inside a frontier AI lab to test a new model for dangerous capabilities. The lab chose you, it pays you and it controls which systems you can see. Your contract says you may not publish confidential information without the lab's consent. You find something worrying. Do you publish it?

Why the

evaluator hesitates The easy answer is "the public must be warned." But the hesitation is real.

• Confidentiality is a genuine obligation. You signed it and the lab has legitimate trade secrets. AI labs fear that embedded evaluators could carry proprietary information to rivals.
• Access is the whole job. As Rumman Chowdhury, former US Science Envoy for AI, put it in the Reuters report, labs control the access evaluators depend on and that gives them leverage over what is reported. Lose the client and you lose the ability to test anything at all.
• Secrets and risks overlap. A person familiar with past dealings told Reuters that what labs consider crucial intellectual property can overlap with what is needed to judge a model's risk. Redacting one can mean redacting the other.

What each side says The Reuters account is that both Anthropic and OpenAI sought significant control over evaluators' publications as late as 5 October.

That came weeks after public statements supporting evaluator independence. Both companies pushed back on parts of it:

• Anthropic said evaluators it has worked with before control their methods, criteria, conclusions and publication. For new evaluator firms, it reserves the right to comment on and approve reports while the working relationship is built.
• OpenAI said it does not seek approval rights over full reports, redacts confidential information as standard practice and lets evaluators disclose whether major redactions were made. These positions are more nuanced than "veto." But they share one feature: the audited party decides what counts as confidential.

The auditor's dilemma we have seen before In 2001, Arthur Andersen was both auditor and paid consultant to Enron.

Its dependence on the client helped it overlook serious accounting fraud and the collapse that followed led to the Sarbanes-Oxley Act of 2002 and stricter rules on auditor independence. The lesson was not that auditors are dishonest. It was that independence cannot depend on the goodwill of the party being audited.

Where the

resolution lies I would resolve this in favour of the public duty, while protecting legitimate secrets:

• Findings on risk should be the evaluator's to publish, without approval rights.
• Genuine trade secrets may be redacted, but every redaction should be disclosed and justified.
• Access and publication rights should come from law or binding public standards, not from contracts the lab drafts. The cost is real: labs may share less or move more slowly. But a voluntary accord that depends on the auditee's editing is not oversight. For the examiner, the point is simple: conflict of interest is solved by structure, not by trust.