The Poisoned Candy In AI Models

According to BleepingComputer, over 100 malicious AI models on the Hugging Face platform provide attackers with backdoors.

Why it matters: Your development team probably uses models from Hugging Face.

• Hugging Face is a website where organizations share AI models. 

• Deep learning models are essentially black boxes; even those who create them do not fully understand how they work, making them ripe for malicious activity by public and private groups.

• In the rush to add AI features, most teams rely upon prebuilt solutions such as those available from Hugging Face.

The big picture: The UK National Cyber Security Centre (NCSC) warns that AI will escalate the threat of ransomware globally.

• James Babbage, director general for threats at Britain's National Crime Agency, explains, "AI services lower barriers to entry, increasing the number of cyber criminals, and will boost their capability by improving the scale, speed and effectiveness of existing attack methods."

Zoom in: Security firm JFrog describes the danger as hiding malicious code within PyTorch's trusted serialization process using Python's pickle module's "__reduce__" method.

• This bypasses security alarms while being installed on machines of the network's most trusted users, developers and data scientists.

Between the lines: Infecting AI models in 2024 is like poisoning candy on Halloween.

• While CTOs and security professionals are busy securing networks by training employees on phishing, attackers create Trojan horses.

Reality check: This is a problematic situation for technical management.

• Policies on what AI models to use and which to avoid are outside most departments' scope.

• Blanket statements that reject all open-source AI models are also out of the question.

• The best solution is to test all new models inside an isolated virtual machine.

Go deeper: Reach out for a free copy of the State of Software report for 2024.