Boards ask how a model performs; the harder question is what it was told to want (Sasha Mudd, Aeon)

Most organisations deploying AI can say what the system does. Fewer can say, in one sentence, what it is optimising for and who chose that.

The philosopher Sasha Mudd makes a useful point about why this gap exists. In a recent Aeon essay she traces much of the AI field back to David Hume's view of reason as a servant: it finds the means to ends it did not set. An AI system, she notes, is judged on how well it optimises goals supplied from outside, against standards it does not set. That framing is attractive because it is clean. You specify the objective, improve performance and reduce error, and the result can be measured, scaled and sold.

What gets lost is the question Kant cared about, which is whether the end is worth pursuing. Mudd argues that this question does not go away when intelligence is defined as optimisation. It passes unnoticed to whoever, or whatever, controls the objective function. Ends that are never debated openly harden into defaults such as profit, speed, efficiency, growth and engagement. In her words, "what cannot be justified openly is smuggled in as architecture.

Her essay is about society, not the enterprise, so what follows is my own extension. A business can run exactly this pattern at small scale. A team picks a metric because it is easy to measure, a vendor ships a default objective, a pilot succeeds against the number, and eighteen months later nobody can say who decided that this was what the system should want. Cost per contact, handling time and conversion rate are all reasonable measures. Each becomes a strategic decision the moment a system pursues it relentlessly.

I think the test is simple to state. For every material AI system, an executive team should be able to name the objective, the person who chose it, the trade-offs that were accepted and the point at which it gets reviewed. If any of those answers is missing, the objective was set by default, and that is still a decision, just one made without anyone owning it.

Boards are right to ask how a model performs. The harder and more useful question is what it was told to want.

Source: Sasha Mudd, "Why real intelligence is something more than optimisation", Aeon (1 October 2026), https://lnkd.in/gXZak4wP

#CorporateGovernance #OperatingModel #Leadership