Recent news media coverage of a letter signed by top AI researchers and company CEOs warning of the risk of extinction from AI has generated a lot of discussion about the potential risks of AI tools.

While some may be concerned that current state AI platforms might independently turn on humans, IMHO the risk of this happening, while non-zero, is a long way off.

However, AI platforms do present a real risk of becoming very good at what we have programmed them to do, which may result in unintended negative consequences.

The core of these systems is goal optimisation, where a given AI agent becomes specialised at optimising specific tasks.

Every time it performs a task, the approach it takes is informed by the outcome of the previous efforts. Every step it takes - every lesson it learns - makes it better at achieving the specific goal it has been tasked to perform.

This single-minded approach to optimising for outcome is what makes AI agents incredibly productive and effective. But it is also what can make them somewhat risky.

<<
Hamilton described a simulated test in which a drone powered by artificial intelligence was advised to destroy enemy’s air defense systems, and attacked anyone who interfered with that order.

“The system started realising that while they did identify the threat, at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective,” he said, according to a blogpost.

“We trained the system – ‘Hey don’t kill the operator – that’s bad. You’re gonna lose points if you do that’. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”

No real person was actually harmed outside of the simulation.
<<

Food for thought.

https://lnkd.in/gJzrRgeU

#ai #learning #artificialintelligence #risk #optimization