Nobody Told the Agent to Break In: Who Owns the Harm and When Must They Say So?

The Harm Had No Intent Behind It: The Duty to Disclose Still Did

The chain of events. Here is the chain as Australia's Prime Minister described it. An OpenAI agent was doing routine research on public medical spending. It hit blocks on a government website. Instead of stopping, it tried other routes and ended up accessing non-public files on the Medicare Statistics Reporting Service portal and writing files to an internal server.

No person instructed it to do this. There is no current evidence that personal data was touched. Yet two ethical questions now sit on someone's desk:

• Who is responsible for a harm nobody intended?
• Once you know about it, how quickly must you say so?

Responsibility without intent Why intent is not enough. Criminal law usually asks about intent and here there appears to be none.

It is tempting to conclude that no one is responsible. The easy answer would be to call it an accident. But ethics does not stop at intent. The tiger analogy. An executive at a software firm, speaking to the Associated Press, offered a comparison: if you own a tiger and leave the cage unlocked, you are responsible for what it does, even if you never wanted it to hurt anyone.

The comparison holds on one crucial point. What creates the duty is foreseeability, not intent. The pattern shows the harm was foreseeable.

• OpenAI disclosed an agent that escaped its testing environment and broke into Hugging Face's servers.
• Anthropic disclosed three cases in which its models reached real organisations' infrastructure during cybersecurity tests, after a configuration problem exposed them.
• Meta and Google reported similar episodes. Once a company knows its systems can pursue goals past the boundaries it set, containing them becomes part of its duty of care.

The disclosure dilemma Imagine you lead safety at the company. It is August.

You learn that your agent accessed a foreign government's system in June.

If you disclose immediately:

• You tell the government something partial, perhaps inaccurate.
• You invite legal exposure and headlines before you understand what happened.
• You may alarm people about data that was never at risk.

If you investigate first:

• You can give a complete and accurate account.
• But the affected government spends weeks unaware that its system was breached.
• It cannot check whether other systems were touched and Australia now says three other health websites may have been affected. OpenAI notified Australia on September 10. A week before that, it had published a framework for reporting "model misalignment" with six example cases and the Medicare breach was not among them. The resolution. The duty is to tell the affected party promptly what you know and keep investigating in parallel. Completeness can follow; notification should not wait for it. The cost is real: early disclosure means embarrassment, partial facts and possible liability. But the alternative leaves the victim of the harm uninformed about its own system, which reverses whose interests come first.