The uncanny valley closed: disclosure and out-of-band verification

The uncanny valley did not close because we finally settled the philosophical question of what makes someone human, but because rendering became fast enough and micro-expressions convincing enough that most people stop examining the interface and simply engage with the person they believe is there.

Tavus claims its Phoenix 4.5 model generates full-frame synthetic humans at 134 milliseconds of audio-to-video latency - a vendor claim, worth treating as such. The direction is not in doubt: realistic synthetic people are moving out of research demonstrations and into ordinary products.

Some will be genuinely useful: a synthetic tutor, guide or service agent can make software easier to use where a text box is a poor substitute for conversation. The difficulty is that the very qualities making these systems effective are the ones making them dangerous, because the trust we extend to a familiar face and voice carries straight through into consequential decisions.

In Hong Kong, an Arup employee joined a video conference with what appeared to be the company's CFO and several colleagues, and made 15 transfers totalling HK$200 million. Every other participant was a deepfake. What defeated him was not a clumsy phishing email but an apparently familiar group of senior people, visibly present, authorising a transaction in the ordinary way.

Disclosure cannot be left to terms of service, a small corner icon, or the goodwill of whoever happens to be operating the system. Where a machine presents as human, the disclosure that it is synthetic ought to be persistent and unambiguous, travelling with the generated image and audio itself rather than being bolted on afterwards by the deployer.

The harder problem is that disclosure-by-design constrains legitimate vendors while criminals ignore it entirely, which is why regulation needs two layers rather than one: mandatory provenance and disclosure for lawful deployment, coupled with out-of-band verification for any consequential action. No payment, credential change or sensitive instruction should become valid merely because a recognisable executive appeared to approve it on a call.

We spent years asking whether synthetic humans could cross the uncanny valley, and they now can, at least often enough that the distinction has stopped protecting us. The question worth arguing about is no longer whether they look real, but whether people can reliably know whom, or what, they are dealing with before trust turns into action.

Directors and executives: what out-of-band check would have stopped those transfers in your organisation?

Sources: Tavus, "Phoenix-4.5" - https://lnkd.in/gXxxjahY | CNN on the Arup fraud - https://lnkd.in/g8fBXMWv