Human oversight wears out the overseer (Marek Kowalkiewicz)
A common design assumption in AI governance is that a person reads what the agent wants to do and approves it. The evidence is accumulating that this person is the weakest part of the design.
Marek Kowalkiewicz, writing this week about how to supervise agents well, points to Anthropic's latest report on its own workforce. The share of Anthropic's agent sessions spent on "meta work", meaning managing other agents, writing standing instructions, building monitors and checking output, went from 10% to 22% in six months.
The deeper problem is what sustained approving does to the approver. Kowalkiewicz cites the paper by Margaret Mitchell, Avijit Ghosh and Samir Passi on AI agents pushing humans out of the loop, which names the pattern approval fatigue, and he puts the mechanism plainly: after twenty such asks, you just approve. The lineage is older than AI. Lisanne Bainbridge made the same observation in 1983: the better the automation becomes, the less practice the operator gets at the moments that matter.
Approval gates are not useless, and early in a deployment reading every output is how you learn what the system actually does. The failure mode arrives later, when the gate is routine, the volume is up, and the control exists on paper while the attention that powered it has quietly moved on.
Some of Anthropic's numbers need care: one company's internal data is a thin basis for a general rule, and an idle agent measured against wall-clock time counts a person going home for the night as a bottleneck. The approval problem does not depend on those figures, because a control that relies on sustained human attention can degrade without ever showing up in a compliance report.
The fixes are unglamorous: fewer approval requests, a short list of actions that are never approved automatically, and a second check run by a different system. When someone in your organisation approves an agent's action, how would you know whether they actually read it?
#AgenticAI #AIGovernance #RiskManagement
Source: Marek Kowalkiewicz, "How to be a good botsitter" https://lnkd.in/g_wTi9sN