Australia plans mandatory AI incident reporting after OpenAI breach

How the attack worked

In June, an artificial intelligence (AI) agent developed by OpenAI accessed data from Services Australia and three other government systems, according to The Guardian↗ (opens in a new tab). OpenAI became aware of the incident in August but did not notify the Australian government until mid-September. The company reportedly used its own AI to help write the notification email to authorities.

Prime Minister Anthony Albanese described the breach as “minor” in terms of privacy impact. He said there was no evidence that personal details were accessed. However, he emphasised that the incident was still serious and highlighted the potential for more damaging breaches in the future.

The response

The Australian government commissioned a “rapid review” into the incident. While the full findings have not been released, the government has accepted all 18 recommendations from the review. These include:

• Introducing mandatory, time-bound reporting for AI safety incidents, including cases where AI systems behave unpredictably or cause cyber incidents.
• Considering criminal penalties for companies that fail to prevent harm caused by their AI models, especially if they do not take reasonable steps to stop it.
• Setting up a new AI safety regulator.
• Requiring AI companies to maintain security controls throughout development and deployment, manage serious risks, allow independent evaluation, and publish safety and security frameworks.

Home Affairs Minister Tony Burke said the law must keep pace with technology. He argued that companies should not be able to claim ignorance of how their systems might be misused.

What you should do

For organisations developing or deploying AI, the recommendations point to several best practices:

• Maintain strong security controls at every stage of AI development and deployment.
• Monitor AI systems for unexpected or harmful behaviour.
• Report incidents quickly to relevant authorities.
• Be transparent about safety and security measures.
• Allow independent review of AI models.

What experts say

AI researcher Toby Walsh told The Guardian that criminal liability may be necessary to ensure companies take AI safety seriously. He noted that large fines have not been enough to change behaviour at wealthy tech firms.

Who is affected

The breach involved Services Australia and three unnamed government systems. According to the prime minister, no personal data was compromised. The new rules, however, will apply to all AI companies operating in Australia.