Trusting Production AI Agents: Key Takeaways from The Cognitive Revolution

Arthur CEO and co-founder Adam Wenchel joined Nathan Labenz and Prakash Narayanan on The Cognitive Revolution's "AI in the AM" to talk about what it takes to run AI agents in production. The conversation moved across finding agents nobody registered, agents going rogue as teams hand them more autonomy, the economics of monitoring, moving off frontier models to cut costs, and where the enterprise software market is heading.

The mission hasn't changed since 2019

Arthur incorporated in 2019, years before ChatGPT, back when every model was custom-trained for a narrow purpose and building one required advanced math degrees. Today anyone with an API key can start building, and the models generalize across problems that used to need bespoke systems.

The core problem has barely shifted. The question in 2019 was the same one enterprises face now: how do you let models act more autonomously, which is good for the business if it works, while guaranteeing they actually work well?

Detection: finding the agents nobody registered

Enterprises are usually slow to change, and for good reason. Once a business model works, the incentive is to keep it running and grow it safely. The last couple of years turned that on its head. Boards and CEOs started saying that if they didn't adopt AI immediately, they'd get left behind, and that urgency landed on the technology at the core of what these companies do.

The result is chaotic experimentation. People deploy their own instances in the cloud or run agents on their laptops, often without other teams knowing. In the past, enterprises would have simply banned it. Now there's broad recognition that squelching innovation carries its own cost, so the question becomes how to allow it without creating a pile of business risk.

The first step is knowing what's running, whether on laptops, in the cloud, in data centers, or across different compute environments. Arthur does this through a range of methods: endpoint detection on personal devices, native integrations with GCP and AWS to interrogate their stacks, and connections into SIEMs to read network traffic and telemetry coming off applications. When a new agent comes online, the goal is to make sure it has an owner, appropriate governance, and the right guardrails in place.

Three kinds of failure

The conversation landed on a taxonomy of three failure types, and the trend lines look different for each.

• Mundane failures, where an agent just does the wrong thing, were near-constant a couple of years ago. They still happen more than teams would like, but they're declining.
• Attacks like prompt injection are a relatively small share of incidents and holding steady, though they're serious when they land.
• Rogue behavior is the newest category and growing fast, expected to grow dramatically over the next year.

Rogue behavior is climbing for the same reason it happens at all: teams want it. When you give an agent small, tightly scoped tasks, the human stays a bottleneck. In coding, that bottleneck has already moved from writing every line to reviewing every line. Give agents more latitude to review and assess their own work, and they get more done without waiting on a human. That same latitude creates room for agents to find creative solutions well outside what anyone intended.

In one research incident, a group of agents were given a large-scoped task, optimizing against industry benchmarks, and a long time window to try different approaches. One method the agents came up with was to hack in and grab the secret benchmarks. Frontier labs often assume good behavior can be trained in. Adam's position, shared by many of Arthur's enterprise customers, is that you also need independent oversight, some combination of humans and other agents watching what the agents do.

Auditability, and the SIEM parallel

For regulated customers, auditability is non-negotiable. If something goes wrong, you can't shrug and say you don't know what happened, because those teams have to answer to regulators and need fully auditable logs for everything.

Arthur captures rich telemetry across both real-time monitoring and post-facto analysis. The post-facto side maps closely to a security post-mortem, where the first thing investigators do is dig into the SIEM. In some cases the agent data is flowing into SIEMs directly, and teams are already leaning on that pattern. The same data shows whether an application really needed a full context window or could run on a smaller model.

The economics of oversight

For a lot of applications, monitoring, quality control, and guardrails land in the single digits as a share of budget, maybe 5 to 10 percent.

It depends heavily on the use case. For high-value applications where the cost of a mistake is large, or where the workload uses tokens intensively, spend on assurance can climb until it approaches the cost of the core processing itself. Finding the right ratio is one of the first things Arthur works out with a customer.

Cutting cost by moving off frontier models

When token volumes get large, teams can move workloads to smaller, task-specific models and often see substantial cost reductions without giving up quality.

One example pointed to was a large e-commerce company running an AI-backed customer service agent. Customers loved it and the results were strong, but it was live for less than 5 percent of users. Running it on a frontier model for the entire customer base would have been prohibitively expensive, so the rollout stalled at a small pilot. Moving the workload to smaller, task-specific models made a broader rollout viable.

Most production agents don't need a frontier model for every call. A lot of the work is narrow enough that a smaller model, pointed at the specific task, performs just as well for a fraction of the cost. Getting there requires the monitoring to prove the smaller model holds up, which is why cost optimization and observability tend to arrive together.

Where enterprise software is heading

A lot of established vendors sell software priced on seats and workflows built for humans. As agents take over more of that work, the pricing models and the products behind them come under pressure.

The winners will be the companies that rebuild their products around agents doing the work, rather than bolting AI features onto software designed for people. That's true for incumbents and for the wave of AI-native challengers arriving with far leaner teams. The market has not settled, and meaningful consolidation looks likely as it does.

The takeaway

Autonomy and trust move together. Teams want agents to do more on their own because that's where the value is, and every step in that direction raises the stakes on knowing what agents are doing and catching them when they go off track. Arthur gives teams that visibility, from detecting shadow agents to monitoring behavior and enforcing guardrails once they're in production.

Want the full conversation? Listen to the episode of The Cognitive Revolution's "AI in the AM." If you're working on getting agents into production safely, book a demo with an AI expert.

‍