Real-time AI guardrails for every model: Arthur on the TrueFoundry AI Gateway

Every team shipping an LLM application runs into the same wall. Developers want to move fast and connect to whatever model gets the job done. Security and risk teams need assurance that nothing toxic, manipulated, or hallucinated reaches a user. Both are right, and the hard part is satisfying both at once, across every model your teams are using, without bolting a different SDK onto every application.

That raises the question most platform teams are asking right now: how do you validate every prompt and every response your AI produces without turning safety into an engineering project of its own?

Today we're answering it. Arthur now integrates with the TrueFoundry AI Gateway. By bringing the Arthur Engine to the Gateway as a custom guardrail, you can validate AI inputs and outputs in real time across all of your models from one place, with consistent policy and a single integration.

What is an AI gateway, and why do guardrails belong there?

An AI gateway is the control layer for enterprise AI. It gives developers and platform teams one place to manage, monitor, and scale their AI applications, with unified access to many large language models, deep observability, and smart routing across providers.

As AI adoption grows, the challenge stops being access to models. It becomes managing everything that comes after: multiple providers, shifting APIs, and strict compliance requirements that can quietly slow teams to a crawl. A gateway brings order to that complexity. It acts as the command center for enterprise AI, unifying access, enforcing security policy, and giving full visibility into every model and environment.

That is exactly why the gateway is the right place to enforce guardrails. Plug them in once at this layer and they apply everywhere. A safety decision made in the dashboard is enforced on every request that flows through, no matter which team, model, or environment sent it.

What the Arthur Engine validates in real time

Arthur sits between your application and your models as a validation layer. The Arthur Engine inspects prompts and completions in real time and returns a clear verdict on whether they are safe to proceed.

Out of the box, Arthur checks for the risks that matter most in production:

• Prompt injection detection. Arthur identifies input crafted to hijack the model and pull it off its instructions, before that input reaches the model.
• Toxicity detection, on the way in and the way out. Arthur flags harmful language so it never reaches your model or your users.
• Hallucination detection. Arthur checks responses against the grounding context you supply, catching claims that drift from the facts before they are trusted as answers.

One detail shapes how Arthur fits in: it is validate-only. Arthur tells you, fast and statelessly, whether something passed or failed. It reports problems rather than silently rewriting text, which keeps its behavior predictable and easy to audit. For teams that answer to a governance review, that distinction matters. You get a verdict you can trace, not a black-box edit.

Better together: one integration, every model

The integration connects Arthur to TrueFoundry through a lightweight deployable wrapper and the Gateway's Custom Guardrail contract. You deploy a small FastAPI service, point the Gateway at it, and register two guardrail configs in the dashboard, one for input and one for output. From that moment, any model behind the Gateway can be validated by Arthur with no per-app code changes.

That is the real payoff. Instead of wiring Arthur into every application by hand, you attach it once at the Gateway. Pin it to a model so every caller is protected, or opt in per request with a single header when you only want it on specific calls. The same policy travels with every team, model, and environment, and every verdict is captured alongside TrueFoundry's request tracking for a clean audit trail.

How the Arthur and TrueFoundry integration works

The integration introduces a layered validation flow that is easy to follow:

1. Your application sends a request, and it first passes through the TrueFoundry AI Gateway.
2. The Gateway forwards the prompt to the Arthur wrapper, which calls the Arthur Engine to check for prompt injection and toxicity.
3. Arthur returns a verdict. An allow verdict lets the request continue to the model. A block verdict halts it, and a safe response is returned instead.
4. Optionally, once the model generates a completion, the Gateway sends that output back through Arthur for a final toxicity and hallucination check before it ever reaches the user.

A subtle but important design choice makes this robust. Policy decisions ride on a normal HTTP 200 with a verdict field, while genuine infrastructure failures surface as 5xx errors. With the recommended "Enforce But Ignore On Error" strategy, a transient outage passes traffic through, but a real policy block always stops the request. For safety-critical rails, you can switch to strict enforcement and fail closed instead.

Where guardrails fit in the agent development lifecycle

Guardrails are the runtime layer of a production-grade AI system. They intercept behavior in real time: pre-LLM checks protect what goes into the model, and post-LLM checks control what comes out. That is precisely what this integration delivers at the gateway.

Guardrails also work best as part of a larger discipline. In our Agent Development Lifecycle, guardrails sit alongside observability, continuous evaluations, and governance to form a complete feedback loop. Observability shows you what your agent did. Evals surface failures across production traffic. Guardrails correct behavior within a single execution, before any response is returned. Together they give security and governance teams the evidence they need to approve a system for production.

That last point is the one enterprise buyers care about most. Being able to demonstrate active, running guardrails is a meaningful signal of production readiness. When a compliance team asks what safeguards are in place, a gateway-level guardrail that validates every prompt and response, with every verdict captured, is a clear and confident answer.

Get started

Securing your AI should not feel like a rebuild. With Arthur on the TrueFoundry AI Gateway, real-time validation is closer to flipping a switch than a wiring project. Deploy the wrapper, add two guardrail configs, attach them to your models, and you are validating prompts and responses across your stack.

To see exactly how it works, head to the Arthur integration guide in the TrueFoundry docs, where the full setup, response contract, and custom check configuration are laid out step by step. If you'd like to see how Arthur fits your environment, book a demo with an Arthur AI expert.

TLDR

• Arthur now integrates with the TrueFoundry AI Gateway, bringing the Arthur Engine to the Gateway as a custom guardrail.
• An AI gateway is the right place to enforce guardrails: plug in once, apply everywhere, across every model and environment.
• Arthur validates in real time for prompt injection, toxicity (input and output), and hallucination against grounding context.
• Arthur is validate-only: it returns fast, stateless allow/block verdicts and reports problems rather than rewriting text, which keeps it predictable and auditable.
• Deploy a FastAPI wrapper, register two guardrail configs (input and output), and attach Arthur to any model with no per-app code changes.
• Policy decisions ride on HTTP 200 with a verdict field; infrastructure failures surface as 5xx. "Enforce But Ignore On Error" passes traffic through on outages, while strict enforcement fails closed for safety-critical rails.
• Guardrails are the runtime layer of the Agent Development Lifecycle, working with observability, evals, and governance to prove production readiness.

‍