One of the more interesting legal questions emerging from the AI era has very little to do with the technology itself - and everything to do with accountability.

As a former lawyer, I have long been uneasy with the black-box nature of software automation. Even before generative AI, organisations were increasingly making consequential decisions through complex software ecosystems where it was often difficult to determine which human - if any - could ultimately be held accountable when something went wrong or the law was breached.

AI is rapidly amplifying that challenge.

Today's AI products are rarely a single system from a single vendor. They are increasingly orchestrations of foundation models, retrieval systems, third-party APIs, agents, workflow platforms and bespoke business logic - often supplied by multiple organisations, integrated by another, and deployed by someone else entirely. The result is an outcome produced by a chain of technologies where no participant has complete visibility, and where human-in-the-loop controls are frequently weak, inconsistent or absent.

The insurance sector offers a good example.

Insurers using AI to underwrite risk are discovering a problem lawyers anticipated years ago. A model can discriminate without ever collecting a prohibited attribute. Postcode, device type, purchasing patterns or browsing behaviour can become highly effective proxies for race, age or other protected characteristics. Regulators are increasingly signalling that proxy discrimination will be judged by its effect, not merely by the variables explicitly collected.

A parallel issue is emerging through AI-washing litigation.

When an AI product is assembled by a decentralised network of model providers, software vendors, implementation partners and internal development teams, who bears responsibility when the system misleads customers or causes harm? The traditional answer - that liability rests with the organisation making the representation - becomes considerably more complicated when the marketing, engineering and operational decisions were made by different parties with different levels of oversight.

Both issues point to the same underlying problem.

We are accelerating AI adoption far faster than we are redesigning governance, accountability and assurance frameworks. Technology has become distributed. Responsibility has not.

Boards are rightly asking whether their AI is accurate, secure and compliant.

An equally important question is whether they can identify the accountable human when it isn't.

Because if your AI vendor's marketing claims outran their engineering, or your own organisation stitched together technologies from half a dozen providers, who is ultimately carrying the legal and regulatory exposure?

That may become one of the defining governance questions of the AI decade.

#AI #Governance #RegTech #RiskManagement #CorporateGovernance