The Three Planes of Enterprise Agent Systems

With Michael Wang, Richter Brzeski, Isaac Chung, Brandon Lei, Daniel He, Braden Holstege, Han Pei, Imani Parker-Fong

Enterprise discussions about agent deployment usually focus on two questions: where the agent runs and whether customer data ever leaves the environment. Both matter, but they describe the deployment boundary rather than how the system changes after it reaches production.

An agent that cannot learn from production behaves much like any other piece of infrastructure. The same failures recur, the same fixes are applied, and the cost of those mistakes remains unchanged over time. That assumption works well for systems like databases, where stability is the goal, but agents are different. Their value comes from observing the outcomes of their work and using those outcomes to improve future behavior. Without that feedback loop, an agent remains fixed at whatever quality it had on the day it was deployed.

We separate the system into three planes because it is solving three different problems. One determines what the agent is allowed to do. One executes the work safely and reliably. One improves future behavior from production signals. Those responsibilities evolve independently, so they should not share the same control plane.

The first is the routing and policy plane, which authorizes every action the agent takes by selecting models, brokering credentials, enforcing tool permissions, applying egress policy, and recording requests. The runtime plane is responsible for execution, providing isolated sandboxes, durable workflows, retries, verification, and production telemetry. The intelligence plane closes the loop by learning from behavioral signals such as verifier failures, retries, latency, cost, and human overrides, improving routing, evaluations, and control logic without requiring customer data to leave the deployment boundary.

The easiest way to see why those responsibilities are separated is to follow a single overnight reconciliation run.

The ticket arrives

A reconciliation job fails inside a financial institution’s VPC. The report still needs to be ready before markets open, even though no operator is online.

The agent inspects scheduler logs, queries the warehouse, compares ledger records, generates diagnostic code, identifies the source of the failure, and prepares a corrected reconciliation for review. Along the way it reaches the wrong conclusion once, receives feedback from a verifier, revises its approach, and produces a correct result for an analyst to approve before the reporting deadline.

From an operational perspective, this is a routine run. The interesting part is not what happens during execution, but the decisions embedded in the system long before the job ever started.

Routing and Policy: The agent requests access

The first thing the agent needs to do is read data from three internal systems.

A straightforward implementation gives the model credentials directly. Service account tokens live in environment variables, mounted secrets, or prompt context and are available throughout execution. That approach works until the model encounters untrusted input. At that point, a prompt injection attack becomes a credential theft attack because the model has access to credentials that can be exfiltrated.

We instead route every request through the routing and policy plane. When the agent needs access to a resource, it requests a capability rather than presenting a credential. The broker evaluates policy, mints a short-lived token scoped to that operation, and returns only the permissions required to complete the task. The underlying credential never becomes visible to the model.

That same request is also the point where model selection happens. At production scale, routing cannot be reduced to a fixed choice. Sending every request to the largest model increases cost and latency without improving many tasks, while always choosing the smallest model gradually erodes quality. The routing decision becomes another policy evaluation that considers the task, latency requirements, customer preferences, and any other constraints defined for the deployment.

By the time execution begins, authorization, credential brokerage, model selection, and egress policy have already been applied. From the agent’s perspective, interacting with an internal system is a single API call. The routing layer absorbs the complexity while enforcing the security boundary.

Execution: The agent writes code

The agent generates a diagnostic script and executes it against production data.

From the runtime’s perspective, it makes no difference whether that code was written by a model or a developer. Every execution is treated as untrusted.

Each run starts in a fresh sandbox with only the resources explicitly granted to that task. There is no ambient network access, no mounted credentials, and no persistent state carried from previous executions. If the code attempts to reach an external service, read a secret it was never granted, or leave artifacts behind for a future run, those operations fail because the environment simply does not expose those capabilities.

Isolation solves only part of the problem. Production systems also have to contend with queue delays, transient failures, process crashes, and network interruptions. The runtime preserves execution state, retries failed steps, records intermediate artifacts, and resumes work from the point of failure rather than starting the entire task over.

Verification is part of execution rather than something layered on afterward. Before an output reaches a downstream system, it can be accepted, rejected, revised, or escalated for human review based on the result of a verifier.

Enterprise deployments also introduce another constraint. Most platforms become multi-tenant somewhere in the stack, which means customer isolation cannot depend on operational processes or developer discipline. The runtime has to enforce those boundaries by construction.

The agent gets the wrong answer

The diagnostic completes and proposes a reconciliation, but one upstream transaction has been counted twice.

No operator is available to catch the mistake. Before the result reaches a downstream system, a verifier compares the proposed reconciliation against an independently computed control total. The invariant fails, the output is rejected, and the agent receives structured feedback describing the discrepancy. Using that feedback, it revises the reconciliation and produces a corrected result.

This is why verification has to be part of execution rather than something reserved for offline evaluation. Offline benchmarks tell you how a system performed on a test suite. Runtime verification determines whether an incorrect result ever reaches a customer or a production workflow.

The runtime records the entire execution alongside standard production telemetry, including tool invocations, retries, verifier failures, warehouse timeouts, and execution traces. Those signals are exported through OpenTelemetry so they appear in the customer’s existing observability platform alongside the rest of their infrastructure.

Much of this architecture will look familiar to anyone who has worked with container orchestration or CI systems. The execution model is similar, but the workload is fundamentally different because the software making decisions is no longer deterministic.

Learning: The run finishes

The report enters a human approval queue. The sandbox is destroyed, temporary credentials expire, and customer data remains inside the deployment boundary.

What leaves the environment is not the execution itself, but a description of how the system behaved. The routing layer learns which model recovered from the task most effectively. The runtime records a warehouse timeout that suggests a retry policy should change. The verifier identifies a reconciliation failure that belongs in the evaluation suite, and human approval confirms that the revised result was correct.

None of those observations require access to ledger contents, prompts, or generated code. They capture the behavior of the system rather than the customer data it processed.

Those behavioral signals become inputs to the intelligence plane. Over time they expand the evaluation suite, improve routing decisions, strengthen verifiers, and refine execution policies. Without them, every run is treated as though it were the first. Routing never improves, retry policies remain static, evaluation suites drift away from production, and verifier quality is frozen at whatever level it reached before deployment.

What crosses the boundary?

Behavioral telemetry is still telemetry. Retry patterns, verifier outcomes, and workflow characteristics can reveal information about how a system operates, so the boundary between customer data and behavioral signals should be explicit.

Customer documents, prompts, model context, credentials, generated code, scratchpads, and application payloads remain inside the customer environment. The only information that leaves is behavioral metadata describing how the execution unfolded, such as the task category, verifier outcome, model identifier, latency, retry count, execution cost, or whether a human intervened.

That channel should be governed like any other production interface. Customers should be able to inspect it, filter it, rate-limit it, or disable it entirely using their existing infrastructure.

Some organizations choose to disable the channel completely. In that configuration, the intelligence plane runs inside the deployment boundary, including fully air-gapped environments. The architecture remains the same, but behavioral improvements arrive through software updates rather than continuously incorporating production observations.

Customer data, credentials, and execution remain inside the deployment boundary in either model. The only difference is how new behavioral knowledge reaches the system.

Questions worth asking

Labels such as SaaS, VPC, BYOC, and air-gapped describe where software runs. They say much less about how the system is designed.

When evaluating an agent platform, we have found a different set of questions to be more informative.

• What information can leave the deployment boundary?
• Can that boundary be verified independently?
• Where do model requests actually transit?
• What production signals influence the system after deployment?
• What changes if those signals are no longer available?

The first three establish how customer data is protected. The last two determine whether the system continues improving after it has been deployed.

The next morning

By the time the operations team arrives, the reconciliation is waiting for review. An analyst approves the result, and the report is delivered on schedule.

Nothing about the deployment boundary has changed. Customer data never left the environment, credentials were never exposed, and execution remained isolated. What changed was the system’s understanding of the work it performed. Routing decisions become slightly better informed, evaluation suites expand to cover another production failure, retry policies become more representative of real workloads, and verifiers improve over time.

Looking back at the reconciliation run, each plane solved a different problem. The routing and policy plane determined what the agent was allowed to do. The runtime ensured the work was completed safely and reliably. The intelligence plane captured what the system learned so the next run could improve. Those responsibilities change independently over the lifetime of a deployment, which is why we keep them architecturally separate.