Probabilistic risks

A model produces text by sampling from probabilities, so it can be steered, and it can drift on its own. An agent also reads text it did not write and can call tools. That makes the risks probabilistic too: no number of test cases proves a bad input will never work, so the aim is to make failures rarer and cheaper, not to rule them out.

Three threats

Threat	How it happens
Prompt injection	Instructions hidden in a prompt, web page or file change what the agent does. OWASP splits it into direct (the user’s own prompt) and indirect (external content the model reads).
Tool call abuse	The agent calls a tool it should not, or with arguments it should not. OWASP files this under excessive agency: too much functionality, too many permissions, too much autonomy.
Data leakage	Private data that sits in context or behind a tool ends up in an answer or in a call. OWASP lists it as sensitive information disclosure.

What is at stake

Four things, and they are why the control patterns later in this trail exist.

• Financial risk: a wrong refund, a wrong order.
• Data integrity: a wrong write, a deleted record.
• Security and privacy: data going to someone who should not have it.
• Reputation: one bad answer, screenshotted.

No single fix

Prompt injection is not solved. OWASP says it is unclear whether fool-proof prevention exists. Anthropic reports a 1% attack success rate for browser agents after training, classifiers and red teaming, and calls that meaningful risk, not a closed problem.

So the answer is layers: guardrailsLLM guardrailsChecks around the model, so a bad input or a bad output is caught by something that does not rely on the model behaving.Opens in a new tab on what goes in and out, least privilege on what the agent can touch (see secure toolingSecure toolingTools are where an agent touches the world, so each one is described, versioned and watched.Opens in a new tab), a person approving the risky actions (human in the loopHuman in the loopA pause in the agent's decision, where a person approves before a high-risk action runs.Opens in a new tab), and a way to stop (ejection logicEjection logicCustom code that stops the agent, or hands over to a person, when a condition is met.Opens in a new tab).