When command and control moves into model APIs

Cisco Talos has examined a Windows implant that asks commercial AI models what to do next, tallies their answers and acts on the winning choice. The panel can include DeepSeek, Qwen, Mistral and Gemini. Once deployed and configured, that loop is designed to run without a human operator issuing each command.

I think the significant change is where command and control moves. An attacker would ordinarily have to operate infrastructure that defenders can find, block or disrupt. Here, the tactical decisions are delegated to ordinary model APIs, used by legitimate applications. The attacker still needs to deploy the implant and supply credentials, but some of the judgement that used to require an operator can be bought as a service.

That dependence is also a point of exposure. Requests to model providers may leave traces on systems the providers run, while refusals, rate limits and account controls can interrupt the loop. That does not establish what each provider can see or whether any has detected this implant. Talos analysed the code and development builds; the public sample has dummy credentials, and Talos has not confirmed deployment in the wild. This is evidence of an architecture, not evidence of an active campaign.

When I read acceptable-use policies, I keep coming back to the less glamorous part of AI safety: whether a provider can enforce its acceptable-use rules against machine-to-machine misuse at scale. A policy can prohibit malicious use on paper, yet that matters operationally only if the provider can recognise suspect patterns, distinguish them from legitimate security work and act quickly enough to disrupt them. That raises questions about what they retain and how they share useful warnings without exposing customer data.

For boards, the lesson is not to put the next cyber incident in a separate "AI risk" box. The APIs staff use for ordinary work can also become part of an attacker's decision infrastructure. That shifts some defensive leverage towards the labs, but it does not relieve an organisation of watching what unfamiliar processes are doing on its own machines.

If you were reviewing your organisation's AI and cyber controls this quarter, what evidence would persuade you that your model providers could spot and stop this kind of misuse, rather than merely prohibit it?

Source: https://lnkd.in/gM7xDGgK

#CyberSecurity #AIGovernance #Risk