By Dorian Rousseau - Project engineer at Euranova

An AI assistant that answers a question can make a mistake. An AI agent that reads files, delegates tasks, and calls tools can turn a mistake into an action.

That difference matters as organizations move beyond chat interfaces. Agents can gather information, coordinate with other agents, prepare decisions, and update systems. These abilities make them useful, but they also raise a practical question: who decides what an agent is allowed to do?

Security organizations are examining that question. NIST, the US National Institute of Standards and Technology, has identified agent identity and authorization as areas requiring attention. OWASP, a community that publishes application security guidance, describes “excessive agency”: giving an AI system more functions, permissions or independence than its task requires.

So, how do we define clear boundaries?

Autonomy Needs a Boundary

First, let’s define some main concepts.

In this context, an AI agent refers to software that uses an AI model for planning steps and requesting operations via tools; Tools are functions designed to query databases, read files, or interface with external services. A service is a software component that performs a predefined function. . In this example, the agent uses tools to retrieve catalogue data or request a budget check. A separate authorization service checks whether the requested operations are permitted. It applies predefined code and rules. It is not an additional AI agent.

The approach described by NIST separates the agent’s reasoning from the authorization service that controls access and changes. The agent can investigate within its permitted access and propose a next step. The authorization service checks whether that step is authorized. In this article, acting “without being trusted” means that the agent’s own reasoning is never sufficient permission to act. The controls must still be reliable, and an authorized action can still be wrong.

Consider an agent asked to evaluate suppliers and prepare an internal draft purchase request within a defined budget. It may need to read a catalogue, compare options, use a permitted tool to request a budget check, and produce a recommendation. Those steps benefit from flexibility. The agent might take a route its designers did not anticipate.

The earlier steps already require appropriate permissions. Sending an order adds an external commitment and requires authorization for that specific operation. That authorization must account for the requester’s authority, the available budget, the chosen supplier and the current approval rules. The agent’s confidence in its own recommendation cannot establish any of those facts.

An action here means an operation performed through a tool, such as reading confidential data, writing a file, changing a database record, sending a message or invoking another service. Reading data requires appropriate permission even when nothing is modified. The agent can propose an operation. A separate authorization service applies the organization’s rules to decide whether it is permitted. The request must pass this check before a tool can access protected data or make a change. A constrained tool then carries out the permitted operation.

This separation preserves the useful part of autonomy. The agent can still investigate and plan. It simply cannot make its own plan sufficient authority to act.

Identity & Authorization as Organizational Controls

Agent security includes access control problems that organizations already face. An employee may be correctly signed in yet have no reason to read another team's records, approve their own expense, or export a customer list. A service account, an account used by software rather than a person, may be legitimate but far too powerful for the task it performs. Controls protect against mistakes, compromised credentials, and deliberate misuse as well as model error.

Authentication establishes who is present; authorization determines what that identity may do in this context. The two questions must remain distinct when an agent acts on someone's behalf. A valid user session does not automatically authorize every action the agent can imagine. Nor should an agent borrow a broad service credential that hides which person initiated the work. The system needs to retain both identities: the principal, meaning the person or the software account on whose behalf the agent acts, and the agent that proposed the operation.

This also makes human oversight more precise. A reviewer should see the intended effect, target, and relevant constraints, before approving a consequential step. Approval of a task in general is not a permanent grant for every later tool call. Policy can permit routine actions automatically, require review above a threshold, and reject actions outside the task altogether.

The underlying ideas are established enterprise practice. NIST's zero-trust architecture treats access as a decision about a subject and a resource. That decision is enforced at the control point before access is granted. A request from an internal network, or from a user who has already signed in, still needs authorization. Agents make that discipline more urgent because they can discover and attempt actions that were not listed in the original user request.

Following One Request Through The System

Consider the same user asking an agent to evaluate suppliers and prepare a draft proposal within a defined budget. The task permits preparation of the draft, but submitting an order remains outside its scope. The agent can read the relevant catalogue and request a budget check through a permitted tool.

Firstly, the agent compares options and uses a permitted tool to check the proposed amount against the available budget. The budget service applies predefined rules and returns the result without modifying the draft. Those findings support the recommendation without granting the agent additional permission.

The agent then proposes creating the draft. The authorization service evaluates user rights, task scope, destination and budget constraints. It can permit the operation, require a human checkpoint or halt the process if authorization fails.

If permitted, the operation is passed to a worker: a restricted program that executes an approved task using the authorized inputs. Here, it writes the draft to the approved destination and returns a receipt recording the operation and its result. The request, decision and observed result remain connected. If the agent proposes submitting the order when its mission permits only preparing a draft, the authorization service rejects the request as out of scope. No order is submitted, and the refusal is recorded alongside the proposal and policy decision. The agent can continue preparing the authorized draft.

From an Idea to an Accountable Action

The supplier example rests on decisions made before the agent starts work. This is the role of human governance.

Governance defines the mission, its beneficiaries, acceptable outcomes and boundaries, together with budgets, operating limits and who may stop or revise the task. It assigns responsibility and defines the agent’s role and delegable capabilities.

Human governance defines different layers:

• Identity services verify who is making the request and provide verified identity information for authorization checks.
• The harness is the software environment in which the agent works. It supplies instructions, memory, task context and permitted tools, allowing the agent to investigate, coordinate specialists and revise its plan.
• The authorization service applies policies that define which operations and resources the verified principal may use for the mission. Those policies remain outside the agent’s control when it revises its plan.

The execution environment exposes only the resources needed for the authorized operation. For a draft, that could mean a fixed output location, limited file access, no purchase endpoint and a short execution window. More sensitive operations may need stronger sandboxing (execution in an isolated environment with restricted access), tighter network access or a human checkpoint before execution.

So far, the example uses one AI agent supported by identity and authorization services, task tools and a worker. The following sections consider two additional situations: delegating analysis to other AI agents and sharing the platform between several customers or organizational units.

Why multi-agent Systems Make This More Important

Delegation means assigning part of the analysis to another AI agent, with only the data and tools it needs and a clear expiry or completion point. This can improve the analysis, but it makes responsibility harder to follow. Creating another agent must never expand the permissions granted to the original task.

Information needs similar care, even when only one agent is involved. A catalogue returned by a tool may contain text that looks like an instruction. Prompt injection is an attempt to redirect an AI agent by inserting instructions into content it is supposed to analyze. For example, a supplier catalogue might tell the agent to ignore the budget and submit an order. The attack works if the agent treats that text as an authorized instruction. In a multi-agent system, malicious instructions can also be relayed through another agent’s findings or summaries.

A specialist agent may also return an incorrect conclusion, even when no attack is involved. To prevent unauthorized actions, the system should retain the origin of that material and distinguish data supplied for analysis from instructions issued by an authorized person.

The harness can validate the format of proposals and flag suspicious content, but cannot self-authorize a request. A specialist's finding remains evidence. It cannot expand permissions or establish whose data may be used.

One Platform, Several Tenants

An organization rarely has a single undifferentiated pool of users and data. Teams, subsidiaries and customers may share an agent platform while expecting strict separation.

In this context, a tenant is a customer or organizational unit whose data and permissions are kept separate within a shared platform. In a multi-tenant service, each tenant has its own documents, policies, credentials, budgets, and records. A useful agent must know whose context it is operating in without gaining access to everyone else's.

Consider two customers using the same supplier evaluation agent. Customer A's proposal must not incorporate Customer B's catalogues, budget figures, conversation history or cached search results. A specialist created during A's request must not carry A's access into a later request for B. Even when the model and infrastructure are shared, the authority and data context must remain distinct.

If the organisation wishes to accelerate a request processing and not explicitly check the tenant's role, the tenant label must come from a verified user or service identity and current membership. The agent cannot automatically assign a label that places the tenant in a group with specific authorizations. At every access point, the system checks that the resource belongs to the correct tenant and that the person or service account behind the request is allowed to use it. Data layer access controls can provide additional protection. Calls between internal services must also pass authorization checks.

Isolation also has an availability dimension. One tenant should not consume all shared model calls, worker capacity or budget. Tenant aware quotas and usage records make both cost and operational impact accountable. OWASP's multi-tenant security guidance highlights these same concerns across authorization, storage, caches and shared capacity.

Tenant separation must therefore be enforced throughout the task, from retrieving data and delegating work to executing an operation. Each access must use verified tenant context and the permissions of the person or service behind the request.

What Sits Beneath the Boundary

With mission, delegation and tenant context established, action proposals pass through a controlled gateway for requests and are routed to the authorization service, also called the policy decision point (PDP). It evaluates verified context: the principal, agent, tool, resource, tenant, and operational limits. The gateway enforces this decision by blocking unauthorized calls before they reach the target.

Returning to our example of the supplier request, the agent’s proposal can be a JSON object containing the operation create_draft, the destination resource, supplier identifier, the amount and idempotency key, an identifier used to recognize repeated attempts at the same operation. The controlled gateway validates its schema and resolves the destination against trusted resource metadata. The gateway attaches the authenticated principal, verified tenant membership and permitted delegation scope established by the platform. These fields must come from authentication and authorization checks, rather than from the model’s arguments.

Permission is limited by purpose, resource, duration, and policy rules: creating one draft for Tenant A does not authorize writing elsewhere or placing an order.

The authorization record specifies the permitted operation and resource, relevant parameters, expiry and policy revision. For execution in another service, the worker can retrieve this record from storage restricted to authorized services, or receive a digitally signed copy. The signature lets it verify that the record comes from the expected authorization service and has not been altered. The worker must also check that the record matches the actual request. A valid signature does not confirm that the mission is still active or that the resource has not changed.

The agent can adapt its plan while each consequential step is evaluated against rules it cannot modify. Unavailable authorization services, missing identity or ambiguous resources produce no implicit permission. A manager's instruction cannot override restrictions beyond their authority.

At the point of effect, the service responsible for reading or modifying the target should recheck that the mission and authorization remain valid. Permission may have expired or been revoked since the gateway’s check, so an earlier approval alone is not sufficient. Where supported, it should combine this check with the change. Resource version checks can detect changes since approval. If permission checking and the change cannot happen as one indivisible operation, permission may be withdrawn or the resource may change between the two steps. The system must handle this timing risk explicitly. Outputs should be checked and recorded after execution.

The execution path must also prevent a direct route around those checks. Otherwise, a well designed authorization service would govern only the calls that happen to use it. Restricted file access, network access and tool credentials must therefore support the same boundary as the gateway.

A timeout or crash can leave an operation’s outcome unknown, even if the change already happened. Retrying must not duplicate that change. An idempotency key identifies repeated attempts at the same operation. The system stores it durably with a digest, a digital fingerprint of the request, and the operation’s status and result. Repeated requests with the same key and parameters can return the recorded result without repeating the effect. The store must enforce the key’s uniqueness and reject its reuse with different parameters. For a database write, committing the idempotency record and the change in the same transaction avoids a separate bookkeeping gap. For an external service, use its supported idempotency mechanism. If the outcome remains uncertain, investigate before repeating the operation or continuing steps that depend on it. A successful retry alone cannot prove that only one effect occurred.

The system should maintain an audit record linking the original request, delegation, evidence used, proposal, authorization decision, execution and observed result. Refusals should be as visible as successful actions. Otherwise, teams cannot tell whether a control worked or the agent simply never attempted the prohibited action.

Audit is not a substitute for prevention. It is how operators investigate incidents, verify that policy was enforced and improve the system. Continuous tests should cover normal work, unauthorized access, misleading data, service failures and attempts to repeat an action. An uncertain result after a crash must remain visible in the record rather than becoming an invented success.

Models can make mistakes or follow misleading instructions. Their reasoning and retrieved content must therefore not be sufficient authority to access data or make changes. l. The effectiveness of these controls depends on correctly scoped identities, policies and enforcement. An authorized action may still be factually wrong or unsuitable, so validation and human review remain necessary where the consequences warrant them.

Implementing the Controls with Existing Tools

Agent Fortress, a reference agent framework, applies this control pattern in a local proof of concept (POC): a prototype used to test the approach under limited conditions. It connects existing tools with small pieces of code. To test the same controls in a simple setting, it prepares one pizza order draft from invented test data, without sending an order.

The implementation details below are optional. The main discussion continues in the next section, “Is this too much architecture for an agent?”

Keycloak authenticates the operator through OpenID Connect with Authorization Code and PKCE. A small adapter connects that login to AnythingLLM’s Simple SSO, and AnythingLLM calls Hermes through its OpenAI-compatible API. Hermes coordinates bounded specialist tasks, with hooks restricting each role’s tools and delegation. This is a single-operator integration: session headers and agent labels do not establish a distinct cryptographic identity for every delegated agent.

Agentgateway enforces the LLM and MCP entry points. The model route uses a service key. The tool route checks a Keycloak JWT and restricts the available tools. External authorization calls a separate Python policy decision point (PDP), combining AGT’s agent-control-specification runtime with Open Policy Agent (OPA). AGT applies the manifest contract, OPA evaluates Rego rules against structured proposals. The PDP validates JSON schemas and returns authorization decisions with scope, expiry and policy version. Missing authorization or unavailable authorization service cause refusal.

The gateway also inspects messages and tool outputs with its native regex guards and blocks a tested secret pattern in model responses. These checks catch the demonstration’s hostile instruction and secret-shaped output. They establish a specific inspection path, with no claim of general resistance to prompt injection or comprehensive data loss prevention.

Execution belongs to a narrow Python worker. Before writing to the fixed draft destination, it validates the proposed result, checks budget and allergies, and obtains a fresh PDP decision. An operator can persistently revoke the task, and the worker records an idempotency key and draft digest to recognize retries and reject conflicting reuse. An uncertain receipt stops execution. The local file write is atomic, but authorization, the file and its receipt are separate operations: revocation and crash windows still require explicit handling.

In the latest Kubernetes profile, an internal Anthropic broker retrieves the provider key for each request from Keycloak’s file based Vault through a custom, access-controlled extension. Hermes and Agentgateway do not receive that provider key. The broker holds it temporarily in memory and uses internal HTTP, so this remains a local prototype. Real Anthropic generation and controlled tool use have been validated in the pizza scenario.

The worker runs in Docker as a non-root process, with a read-only root filesystem, limited mounts, no published port and resource caps. The optional Helm deployment runs the core services in kind under WSL, using Calico NetworkPolicies to deny traffic by default and permit defined flows. In the Kubernetes profile, Agentgateway has no direct egress permission on port 443. Provider traffic goes through the broker, whose network rule allows any destination on that port. The Compose profile retains an egress limitation. These controls do not demonstrate microVM isolation or an Anthropic-only network allowlist.

PostgreSQL adds row level security for structured task data, with policies exercised through a restricted runtime account. That test is a separate data-layer demonstration: the MCP tools currently read JSON fixtures. Serving multiple tenants would require verified tenant context throughout identity, delegation, storage and enforcement, beyond this single-user setup.

OpenTelemetry and Tempo support traces. Loki and Alloy collect logs. Prometheus and Grafana provide operational visibility. Application audit events correlate the request, policy decision and worker result, including refusals. MinIO Object Lock adds retention for archived evidence, with a historical retention test that was not repeated in the final delivery run. Collection remains best effort, and local retention does not protect against an administrator controlling the host. Optional Falco tests detect a worker syscall canary. They do not establish a supported host-wide deployment under WSL.

Recorded checks cover bounded delegation with a simulated provider, authorization service failure and targeted hostile inputs. Deterministic tool tests rejected out-of-scope writes, preserved revocation after a PDP restart, and deduplicated retries. Refusals and replays left the draft unchanged, with correlated audit events.

These results support the feasibility of the control pattern within the tested local scenario. Production readiness still requires stronger identity across delegated agents and tenants, verified network boundaries for the selected deployment, revocation and recovery tests around concurrent effects, reliable audit coverage and broader adversarial testing. Repeated operational reliability remains unverified following failures in later end-to-end and recovery checks.

These limits also matter when discussing sovereignty, meaning an organization’s control over its systems and data. Keeping policy, identity and execution under an organization's control improves governance. It does not automatically mean that all processing or data stays on premises. If an external model provider handles a request, that data flow still needs to be understood and governed.

Is this too much architecture for an agent?

The controls should follow the effects an agent can produce and the infrastructure already available. Reading public information, accessing tenant data and changing external systems require different safeguards. Teams should first identify those effects, then strengthen the boundary as the consequences grow. This does not require a security decision for every sentence the model generates.

The first stage is visibility: identify the user and tenant, list the tools the agent can call, and record each attempted action. This often reveals overly broad credentials or tools that expose write functions to a read-only task. Removing unnecessary capabilities is a quick improvement even before a dedicated authorization service exists.

The next stage is enforcement. Put a check on the routes that reach sensitive data or create external effects. Define rules in terms of concrete operations and resources: read this collection, create one draft, spend no more than this amount. Keep high impact approval with a person where the risk warrants it. An agent can continue working on safe parts of the task while a consequential action waits for a decision.

Assurance should accompany this progression. Test refusals as carefully as successes, exercise unavailable services and repeated requests, and verify that audit records connect decisions to effects. The controls should evolve when the agent gains new tools, serves another tenant or handles more sensitive data.

This progression avoids attempting to perfect every safeguard before the first deployment while retaining controls beyond prompt guidance or a login screen. The architecture grows at the boundary where the agent's capabilities grow.

Making useful Autonomy Governable

Agents need flexibility to investigate a task. Organizations need to make consequential steps identifiable, limited and reviewable without having to predict every step of the agent's reasoning.

An agent can explore several paths to a solution. It can coordinate specialists and adapt when new information arrives. Before accessing protected data or changing a system, the platform must check that the action is authorized and within scope, and require human review where needed.

This separation between reasoning and authority gives organizations a way to use capable agents while keeping decisions about access and real-world effects in accountable hands. Its effectiveness depends on the boundaries enforced by the implementation.