Cybersecurity researchers have discovered that it is doable to compromise the Hugging Face Safetensors conversion service to in the end hijack the fashions submitted by customers and end in provide chain assaults.
“It is doable to ship malicious pull requests with attacker-controlled information from the Hugging Face service to any repository on the platform, in addition to hijack any fashions which might be submitted by the conversion service,” HiddenLayer mentioned in a report printed final week.
This, in flip, could be achieved utilizing a hijacked mannequin that is meant to be transformed by the service, thereby permitting malicious actors to request adjustments to any repository on the platform by masquerading because the conversion bot.
Hugging Face is a well-liked collaboration platform that helps customers host pre-trained machine studying fashions and datasets, in addition to construct, deploy, and prepare them.
Safetensors is a format devised by the corporate to retailer tensors preserving security in thoughts, versus pickles, which has been possible weaponized by menace actors to execute arbitrary code and deploy Cobalt Strike, Mythic, and Metasploit stagers.
It additionally comes with a conversion service that allows customers to transform any PyTorch mannequin (i.e., pickle) to its Safetensor equal through a pull request.
HiddenLayer’s evaluation of this module discovered that it is hypothetically doable for an attacker to hijack the hosted conversion service utilizing a malicious PyTorch binary and compromise the system internet hosting it.
What’s extra, the token related to SFConvertbot – an official bot designed to generate the pull request – may very well be exfiltrated to ship a malicious pull request to any repository on the positioning, resulting in a state of affairs the place a menace actor might tamper with the mannequin and implant neural backdoors.
“An attacker might run any arbitrary code any time somebody tried to transform their mannequin,” researchers Eoin Wickens and Kasimir Schulz famous. “With none indication to the person themselves, their fashions may very well be hijacked upon conversion.”
Ought to a person try and convert their very own personal repository, the assault might pave the way in which for the theft of their Hugging Face token, entry in any other case inner fashions and datasets, and even poison them.
Complicating issues additional, an adversary might make the most of the truth that any person can submit a conversion request for a public repository to hijack or alter a broadly used mannequin, doubtlessly leading to a substantial provide chain threat.
“Regardless of the very best intentions to safe machine studying fashions within the Hugging Face ecosystem, the conversion service has confirmed to be weak and has had the potential to trigger a widespread provide chain assault through the Hugging Face official service,” the researchers mentioned.
“An attacker might achieve a foothold into the container working the service and compromise any mannequin transformed by the service.”
The event comes slightly over a month after Path of Bits disclosed LeftoverLocals (CVE-2023-4969, CVSS rating: 6.5), a vulnerability that permits restoration of knowledge from Apple, Qualcomm, AMD, and Creativeness general-purpose graphics processing items (GPGPUs).
The reminiscence leak flaw, which stems from a failure to adequately isolate course of reminiscence, permits a neighborhood attacker to learn reminiscence from different processes, together with one other person’s interactive session with a big language mannequin (LLM).
“This information leaking can have extreme security penalties, particularly given the rise of ML techniques, the place native reminiscence is used to retailer mannequin inputs, outputs, and weights,” security researchers Tyler Sorensen and Heidy Khlaaf mentioned.
