“The AI mannequin powering the agent have to be handled as an untrusted element,” the researchers wrote within the paper, warning that “semantic guardrails” and prompt-level defenses alone can not reliably safe programs as soon as brokers acquire entry to enterprise instruments, reminiscence, APIs, browsers, and execution environments.
The authors drew the comparability to working programs. “Just like how an working system treats a course of as untrusted, we take the stance that the mannequin powering the agent must be handled as untrusted and security properties must be expressed and enforced outdoors, on the stage of the surrounding system,” they wrote.
The paper was written by researchers at Google, the College of California, San Diego, the College of Wisconsin-Madison, and different establishments, together with Mihai Christodorescu, Earlence Fernandes, and Somesh Jha.
5 ideas from programs security
The authors distilled 5 ideas from many years of programs security analysis that they mentioned agentic programs ought to comply with: least privilege, tamper resistance of the trusted computing base, full mediation, safe info circulate, and accounting for the human as a weak hyperlink.
