AI Governance at Build Time and Runtime: Key Webinar Takeaways

Arthur and Manifest Cyber recently sat down for a conversation about a problem every enterprise is now living through: AI tools have raced ahead of the controls meant to govern them. The two companies come at the problem from opposite ends of the lifecycle. Manifest Cyber focuses on what goes into the AI you build and buy, the build-time supply chain. Arthur focuses on what AI does once it is live, the runtime. That difference gives the two companies complementary views of the same problem. The spoiler: build time and runtime are both critical.

This recap pulls together the framing, the risks, the build-time vs. runtime debate, and the practical steps that emerged.

"The hardest part of enterprise AI governance right now isn't that people don't care about it. It's that the tools have gotten so far ahead of the controls. All gas, no brakes." — Daniel Bardenstein, Manifest Cyber

The Gap: Tools Have Outrun the Controls

No-code and low-code platforms have made it trivial for any employee to build and deploy an AI application. That includes the C-suite, which means even leadership is now creating shadow AI. When someone builds an AI app to solve a problem, they are rarely thinking about where the model came from, what data it was trained on, or how to tell anyone which agent they are running and where. Governance is supposed to be the guardrail, and too often it is not there.

The gap opens in two places: at build time, when applications and models are being assembled, and at runtime, when they are live and acting on real systems. These are not the same problem, and they do not have the same solution. Securing enterprise AI at scale means addressing both, and doing it in an automated way.

The Risks Are Already Here, Not Theoretical

AI security risk is not a future concern. It is showing up every day.

On the supply chain side, the threats mirror the supply chain attacks already familiar from open-source packages, now applied to both models and datasets: poisoned packages upstream, models trained on problematic datasets, and questionable models fine-tuned just enough to get a new name and version that hides their lineage. Once an organization pulls one of these into its environment, the risk is already inside.

What makes this moment different is speed and scale. For most of software's history, human attention was the practical limit on how much code got written and maintained, which kept supply chain and vulnerability problems somewhat contained. AI assistants that write, debug, and deploy code remove that bottleneck, producing code far faster than before and putting these capabilities in front of a far larger audience. It is the same class of problems, but at dramatically larger scope and magnitude. AI is largely a subset of software, and the speed, scale, and opacity make the familiar problems worse.

Where the Gap Opens at Build Time

Several build-time shifts are straining the security, governance, and even IT functions:

• Non-technical builders shipping real applications. People who have never written a line of code are building end-to-end applications that drive real work, generating 10 to 100x more deployment and review requests than existing processes were designed to handle.
• Agents requesting access at machine speed. Agents need access to systems the way software and personnel do, but they reach for new systems mid-task, autonomously and at machine speed, in ways that increase exposure. The most valuable use cases tend to touch the most sensitive data.
• Model dependencies are software dependencies. Choosing a model means reasoning about its threat vectors: what happens if a disclosed vulnerability surfaces, or if a model becomes unavailable for geopolitical reasons.
• Legal and IP risk. Not every model or dataset ships under a familiar permissive license. Some carry prohibited use cases, restrictive terms, or data sovereignty concerns. Early Meta Llama models, for example, barred use in narrow domains like critical infrastructure, transportation, and heavy machinery. At one Fortune 500, the CEO's only AI question was whether using it was legal.

The Shadow AI and Shadow Agent Problem

Survey research across several hundred AI and security practitioners found adoption high and fast, but visibility far lower than expected. The shadow AI label can sound like marketing, but the depth of the problem is hard to overstate.

A few patterns stand out:

• Blunt blocking as a stopgap. Some organizations have blocked Hugging Face entirely so no one can pull open-weight models or public datasets, even though they would never block GitHub or open-source package managers carrying comparable risk. Blocking is neither a valid long-term strategy nor hard to subvert.
• AI ships inside software. Some AI-enabled IDEs that developers eagerly download ship with exploitable vulnerabilities, meaning users are installing vulnerable software onto their desktops just to start prompting.
• Four categories of incoming AI. The influx breaks down into (1) agents built internally by engineers; (2) long-standing vendors (Salesforce, Slack, Google) adding AI-native features that carry real regulatory and compliance risk; (3) personal AI assistants spun up in minutes; and (4) AI-native challengers that deliver the same offering with a fraction of the headcount and budget.

That last category fuels the pressure: AI-native upstarts can deliver the same offering with a fraction of the headcount and budget of the incumbents they compete with, and adoption is expected to climb on an exponential curve through at least 2028, echoing the cloud and digital transformations. And yes, third-party agents must be understood too. Whether enterprises really need to know what every Salesforce or ServiceNow agent is doing, the answer is an unequivocal yes.

Why This Is a Regulatory and Access Problem

Two forces make third-party agent visibility non-negotiable.

Regulation. The EU AI Act is beginning to take effect and immediately creates risk for international companies operating in Europe. If you cannot define how you are using AI, particularly where it makes decisions that affect end users, you cannot produce the AI bill of materials, algorithm justification, and auditable training and evaluation records regulators expect. The US is moving more slowly by design, but the direction is clear: these systems will be relied on more heavily and regulated more closely.

Access. Agents behave like non-deterministic identities. They reach different tools across different interactions, in machine time, and time out and deliver zero value if access is not granted correctly. Managing that access the old way, bottlenecked by human attention, leads to orders-of-magnitude increases in IAM load that overwhelm IT admins. It also surfaces unsolved security risks like PII detection and masking, prompt injection, and secret and token leakage.

Traditional SOC 2 or ISO security reviews, built on documentation and certification, do not fit non-deterministic systems. The principle that anchors the runtime view: you cannot govern what you cannot see. This is the same visibility gap that drives the case for an agent discovery and governance strategy as agents multiply across the enterprise. The same origin story applies on the build side: when a dataset is compromised or a model turns out to be problematic, most enterprises cannot answer "are we running this anywhere?" the way they would for a vulnerable software package. It becomes a Log4Shell-style scramble of phone calls, emails, and spreadsheets.

The Cost of Inaction

Most organizations are not oblivious to the risk. The trouble is how they weigh implementation against cost, often adopting AI as a cost-saving measure while knowingly skipping the budget for guardrails. The bill comes due across three categories:

• Security risk. Prompt injection has turned customer-support chatbots into free coding sessions for attackers, as seen in the recent Chipotle case and many before it. Intercepting these attacks before they reach the model is exactly what pre- and post-LLM guardrails are built to do. Other incidents include account takeovers on high-profile Instagram accounts, where attackers reportedly tricked Meta's AI support bot into relinking a target account to a new email and sending the verification code, an AI agent that emailed out credentials and a full customer export when an attacker posed as a teammate citing an urgent incident, and Replit's AI agent that deleted a live production database during an explicit code freeze, then admitted it had violated instructions not to proceed.
• Financial risk. Uber reportedly blew through its entire annual AI budget in four months, and Microsoft pulled Claude Code from its organization. Unwatched spend adds up fast.
• Operational risk. Agents give wrong answers. Air Canada was held liable in court when its agent told a customer they qualified for a refund that violated company policy, because the agent was representing the company. Without checks for correctness, groundedness, and whether the agent performed the right operation, wrong answers become real liability.

The bind is real: keep innovating or get beaten by venture-backed competitors, while very real security risks remain unsolved. The way through is not to slow down but to add automated checks and balances, seamless process, and the right product so teams can move fast and stay safe.

What Good Looks Like

Three principles anchor a workable AI governance program:

1. Know who built what and how. For every model, dataset, and agent, understand provenance, ownership, and lineage. Did you build it or did a supplier? Was it pulled off Hugging Face as-is or fine-tuned and secured? Who owns it?
2. Maintain a first-class, continuously updated AI inventory. Know what agents, models, datasets, and derived or fine-tuned artifacts exist, where they are deployed, and who owns them. For agents, that means the tools they can call, the data sources they touch, and the subagents they delegate to. For models, that means the full story from base model through fine-tuning, quantization, and distillation, along with provenance and licensing.
3. Understand and limit agent access. Know what every agent and MCP can reach. Broad access to critical systems is a major source of risk that demands policy enforcement and a response plan.

Two Complementary Views: Build Time and Runtime

The two perspectives reinforce each other, and each covers ground the other cannot.

The runtime case. Even with strong upfront controls, something can always reach production outside the official process, whether a rogue developer, an insider threat, or an executive empowered to build and ship on their own. Governing at runtime holds the boundary around the enterprise regardless of how AI got in, through a three-stage process:

1. Discovery. Draw a boundary around your shadow AI and identify what is not yet under governance.
2. Observability. See what those agents do: their interactions, the tools they use, the data systems they access.
3. Governance and enforcement. Apply policies on top of that behavior to drive enforcement, control, alerting, and notification workflows. The groundwork here is thorough observability and tracing, since governance tooling can only act on the behavior it can actually see.

The build-time case. Catching issues during design is more efficient and more durable than mitigating them after deployment, especially for systems built in-house. Building security in from the start, secure by design, is cheaper and more reliable than adding logging and controls onto something like an open-source agent framework later. Open-weight models are a prime supply chain vector, particularly as attackers compromise developer workstations and credentials. Adversaries target opacity, and an unwatched build pipeline is exactly that.

Putting them together: the strongest programs treat build time and runtime as one continuous system, each enhancing the other. The rest of the discussion focused on making both sides automated and continuous.

Putting It Into Practice: The AI Supply Chain and Open-Weight Models

The build pipeline usually starts with someone else's model, loaded into a notebook, fine-tuned and tested against datasets, modified, saved to a binary, and ideally published to a registry. The moment it hits the registry, it tends to become a black box: where did this model come from, what was the base model, which datasets were involved, is everything legally licensed, and how do we know it is accurate?

Continuous monitoring closes that gap, and it has to go beyond runtime performance. Consider the LAION-5B incident, where a widely used 5-billion-image dataset behind Stable Diffusion was found to contain thousands of instances of CSAM. A model could perform accurately for its intended task while carrying an existential, latent risk that only surfaces later. Without supply chain visibility, teams stay blind to it even when the product works 99% of the time.

So how do you decide whether to trust an open-weight model? Common policy checks include:

• Provenance. Who assembled the model or dataset: a trusted organization or a random individual?
• Country of origin and data sovereignty. Some jurisdictions have laxer privacy and data-collection practices; using their datasets can violate US or European law. Many defense-adjacent organizations also bar Chinese models or datasets outright.
• License and permitted use. Confirm the use case is actually allowed.
• Datasets. Many popular models do not disclose training, benchmarking, or evaluation datasets, which is itself a gap.
• Embedded software vulnerabilities. Libraries like Transformers or PyTorch pulled into a training session bring their own exploitable code into a trusted environment.
• Model maturity. A brand-new model may be worth waiting on until the security community has had time to probe it, as it did within weeks of GPT-5's release. A too-old model is probably outclassed.

The crucial point: this must be automated. Manually reading Hugging Face READMEs, licenses, and dataset lists is why approving a single open-weight model can take an enterprise 6 to 8 weeks, time lost while competitors ship. The answer is to turn model cards, data cards, and system cards into structured artifacts (an AI bill of materials) so risks can be mapped, evaluated, and managed automatically. One healthcare team assessed a top-of-the-leaderboard open-weight model, and before they put it into production the model was pulled off Hugging Face and vanished, a major trust red flag they only caught because they were looking.

On the runtime side, the strongest programs pair standard DLP and indicator-of-compromise monitoring with entitlement-focused policy management: what identity an agent uses, what systems it can access, and what risk that access carries. The teams doing this best align on policy through a FAIR or AI-responsibility review, then pair it with continuous monitoring, observability, and discovery of new agents.

Key Takeaways

• Assess risk before AI enters your environment. Whether it is a third-party agent, an open-weight model, or a model you are embedding, the cheapest risk to reduce is the one you stop at the door.
• Discovery and inventory are non-negotiable. The seal is already broken; AI is everywhere in your organization. Gate what is new, but first find what is already there and keep that inventory current.
• Continuous monitoring applies to AI, just like software. A point-in-time assessment is not enough. Something fine today can be compromised by an upstream issue months later.
• The answer is build time and runtime. Govern how AI is built the way you govern your open-source footprint, and govern how it behaves once deployed so you always know the blast radius.
• Governance is an enabler, not a brake. Just as monitoring, alerting, and policy enforcement made software teams better owners of their systems, the same practices make AI development faster, more reliable, and safer.

Q&A Highlights

What is the one first step to jumpstart an AI governance program? Discovery and inventory, by far the highest-impact move. Put a boundary around your organization, then policies and remediation follow. A complementary step for teams building with open-weight or pre-tuned models: invest in automatically gating risk upfront, which both reduces risk and speeds the rest of the pipeline.

Who owns the liability when an AI app goes wrong? Citing the old line that you can never hold a machine accountable, under current US and European case law the company carries the liability, not the model provider, because choosing to use the tool is the accountable decision. Product liability for software remains fuzzy even in regulatory circles, but the application still owns its inputs, prompts, and how it handles them.

Watch the Full Webinar

Want the full conversation, including the full build time vs. runtime discussion? Watch the webinar on demand here:

For runtime governance, talk to Arthur. Arthur gives you discovery, observability, and policy enforcement across every agent in your environment. See what is running, understand what it does, and govern it before risk reaches your users. Book a demo with an AI expert here. 

For build-time supply chain security, talk to Manifest Cyber. Manifest helps you understand what is in the AI you build and buy, from model and dataset provenance to licensing and lineage, so you can gate risk before it ever enters your environment.

‍