Maybe we are getting closer to AI doing a perp walk

Maybe we are getting closer to AI doing a perp walk.

Spain's data protection regulator has received what it says is the country's first reported notification of a personal-data breach allegedly carried out by an AI agent. According to the affected organisation's report, the agent used a widely known large language model to find a vulnerability, gain access to a system, modify personal data and view invoices, with limited human intervention across the attack chain.

The caveats matter. The AEPD says the facts are still under review, the organisation and model have not been identified, and one case does not establish a trend. It also stresses that using a particular model does not mean the model itself, or its provider's infrastructure, was compromised. This is not evidence that a chatbot escaped and turned criminal.

Yet it may be the point at which an unmanaged AI moves a CEO closer to a perp walk.

The legal system does not need to decide whether an agent had intent, understood the consequences or can be blamed. It already knows where to look: the organisation that deployed the system, the people who authorised its access, and the executives responsible for the controls around it. An AI agent can discover vulnerabilities, select actions and alter data, but it cannot front a regulator, lose its licence, answer to a board or stand before a court.

That changes the governance question. The familiar reassurance that there was a "human in the loop" is almost beside the point once the agent can execute a multi-stage action faster than the human can see, understand or stop it. The real questions are who gave it authority, what tools and data it could reach, whether its actions can be reconstructed, and which named person bears the consequence when the controls fail.

The AEPD's guidance on agentic AI is useful precisely because it treats agents as systems embedded in organisational decisions: their behaviour reflects their design, deployment, configuration, objectives, available information and access to tools. Autonomy does not dissolve accountability. It concentrates it around the people who decided that autonomy was acceptable.

AI still cannot do a perp walk. The emerging risk for executives is that it can now create the evidence trail that sends somebody else on one.

Reuters: https://lnkd.in/gsVMTAqG

#AIGovernance #CyberSecurity #DataPrivacy #AusBiz