Agent Risk 101 for CISOs: Why AI Agents Break Your Security Model

Your security model was built for two kinds of software. The first is deterministic code: it does what it was written to do, and you can read the code path to know what that is. The second is machine learning models that make predictions but don't act on them. AI agents are neither. They reason, plan, and take actions across your systems, and they do it non-deterministically, so the same input can produce different behavior on different runs.

That shift is why agents land on the CISO's desk as a new problem rather than a variation on an old one. An agent with tool access and standing credentials is closer to an unpredictable insider than to an application. This primer defines the category: what makes agent risk distinct, where it shows up, why existing controls miss it, and what it takes to bring agents under one governance framework. A follow-up will cover how to govern agent risk in depth.

Why agent risk is a new category

For most of the last two decades, application security assumed a readable code path. You could trace an input to an output, review the logic, and reason about failure. Machine learning added probabilistic behavior, but classic ML models still sat behind an application that decided what to do with their predictions. A fraud model scored a transaction; deterministic code decided whether to block it.

Agents collapse that separation. They decide and act in the same loop. Give an agent a goal, a set of tools, and access to your data, and it will choose which tools to call, in what order, using context it assembles at runtime. The intelligence that makes agents useful is the same property that makes them hard to secure: their behavior is emergent, not enumerated. You did not write every path the agent can take, so you cannot review them all in advance.

The practical consequence for a CISO is blast radius. A chatbot that only answers questions can embarrass you. An agent that books refunds, updates records, queries a production database, or calls internal APIs can cause direct operational and financial damage before anyone notices.

What makes agents risky

Four properties drive most agent risk. Naming them helps because each maps to a different control gap.

Autonomy. Agents take multi-step actions without a human approving each step. That is the point of using them, but it means a wrong decision propagates through several actions before a person can intervene.

Tool and system access. Agents reach real systems: APIs, databases, internal services, and often other agents. These integrations are frequently over-permissioned, granted broad access during a prototype that never got scoped down before production.

Non-determinism. The same prompt can produce different behavior across runs. Testing an agent once tells you how it behaved that once. Reliable assurance requires evaluating behavior continuously, not signing off after a single test pass.

Emergent behavior. Agents exhibit capabilities and failure modes nobody explicitly designed. A prompt change to fix one behavior can introduce a regression somewhere else, which is why agent development looks more like an experimental loop than a linear build.

The shadow agent problem

Last year's worry was shadow AI: employees pasting sensitive data into public chatbots. The 2026 version is shadow agents, and it is harder to contain because agents enter the enterprise from every direction.

They arrive through new application development, as engineering teams add agents to almost every new project. They arrive through third-party solutions built on agents for legal, finance, and customer service. And they arrive, most quietly, through vendors bolting agents into software you have run for years, so an existing tool starts making autonomous decisions after a routine update. The result is sprawl across Vertex AI, Bedrock, Agent Foundry, LangChain, and more, with no central inventory of what is running where.

McKinsey reports that 80% of organizations already see risky behavior from AI agents. The first-order problem behind that number is visibility: you cannot secure what you cannot see, and manual spreadsheets do not keep pace with agents that multiply from dozens to thousands across an enterprise.

The core agent risk categories

CISOs and their teams tend to ask "what can actually go wrong." Here are the categories worth knowing by name.

Data leakage and PII exposure. Agents assemble context from many sources and send it to model providers. Without controls, sensitive personal or proprietary data can leave your environment inside a prompt, or surface in a response to the wrong user.

Prompt injection and instruction hijacking. Malicious input, hidden in a document, a web page, or a user message, can override the agent's system prompt and redirect its behavior. This is the most common attack pattern against agents, and it targets the reasoning layer rather than the infrastructure.

Excessive agency and over-permissioned access. An agent granted broad read/write access can take actions well beyond its intended job. When permissions are too wide, a single bad decision reaches systems it never should have touched.

Hallucination and unsupported claims. Agents state facts not supported by the context they had. In a customer-facing or clinical setting, a confident wrong answer is a compliance and trust problem, not just a quality one.

Toxic or off-brand output. Harmful, inappropriate, or off-brand responses carry reputational and legal risk. Toxicity has subtle forms too: what is appropriate for a hospital intake agent is not appropriate for an airline support agent.

Unauthorized or unintended actions. Agents call the wrong tool, take an action the user never asked for, or execute a step out of sequence. Because the action is real, so is the consequence.

Untraceable behavior. An agent that emits no telemetry is invisible. When something goes wrong, there is no trace to reconstruct what happened, which makes both incident response and audit impossible.

Why traditional tooling falls short

Legacy monitoring and MLOps tooling were built for a different shape of system. Application security assumes deterministic code paths to review; agents do not have fixed paths. ML observability assumes a model that scores inputs, not one that plans and acts across tools. Static analysis has nothing to read, because the agent's "logic" is generated at runtime from prompts and context.

Manual approaches fail for a different reason: scale. Self-reported inventories and periodic reviews cannot keep up with thousands of agents that change behavior with every prompt update and appear without going through a governance gate. By the time a quarterly review runs, the environment has already moved.

What managing agent risk actually requires

Bringing agents under control takes a few capabilities working together. This is the preview; the governance follow-up goes deeper on each.

Start with discovery. You need continuous, automated detection across every environment, not a spreadsheet, so agents get found regardless of how they entered. From there, every agent needs an accountable owner, so nothing runs without someone responsible for its behavior and compliance.

Then come the runtime controls. Guardrails intercept behavior in real time, redacting PII before it leaves your environment, blocking prompt injection, and catching hallucinations or toxic output before a user sees them. Those same guardrails work best as a self-correcting loop, where a flagged response is fed back to the agent for revision before anyone sees it, an approach worth understanding when you set expectations for how agents should self-correct. Continuous evaluation and monitoring watch production traffic for drift and failures, so problems surface before a customer reports them. Arthur's agent security and governance (ASG) platform ties these into one framework that serves the CISO, audit, and governance functions from a single source of truth, which matters because those roles have historically shared a blind spot rather than a shared view.

How Arthur fits

Arthur's ASG platform follows a simple path: discover every agent across telemetry, MCP servers, network traffic, and cloud APIs; triage and rank them by risk; onboard each into a governed application with an owner; then govern with guardrails and continuous monitoring. The data plane runs inside your own VPC, so telemetry and sensitive data never leave your environment while the control plane gives you org-wide oversight.

For security, that means a full inventory and risk scoring across environments. For audit and compliance, it means immutable logs, clear ownership, and enforced policy mapped to controls like SOC 2, RBAC, and SSO. For the governance office, it means running one program across thousands of agents without adding headcount for every new one.

TL;DR

• Agents reason, plan, and act non-deterministically, so they break a security model built for deterministic apps and predict-only ML models.
• Four properties drive the risk: autonomy, tool and system access, non-determinism, and emergent behavior.
• Shadow agents enter from new apps, third-party tools, and vendor updates, sprawling across clouds with no central inventory. McKinsey reports 80% of organizations already see risky agent behavior.
• The core risk categories: data leakage and PII exposure, prompt injection, excessive agency, hallucination, toxic output, unauthorized actions, and untraceable behavior.
• Legacy monitoring, MLOps, and manual review cannot see or scale to autonomous, tool-using systems.
• Managing agent risk starts with discovery and ownership, then adds guardrails and continuous evaluation under one governance framework.

Next steps

See how Arthur discovers and governs every agent in your environment. Book a demo with an AI expert, or explore the Agent Development Toolkit if your teams are building agents and want observability, evals, and guardrails from day one.

This primer is for general information and is not legal or compliance advice.

‍