Decision, Authority, Execution: Observability for AI Agents

"The model said refund requested" is not enough information to operate a customer-service agent. An operator needs to know what the system decided, whether authority existed, and whether a business write was attempted.

Those are three separate facts:

Decision     require_confirmation
Authority    not_granted
Execution    not_attempted

The first describes the control-plane decision. The second describes the permission boundary at that point in the workflow. The third describes what happened to the business action. Combining them into one status makes incidents look simpler than they are.

Model output is only the beginning

The model can propose an intent such as "refund this order." That proposal still needs authentication, customer scope, authoritative target resolution, typed validation, policy, and confirmation. A proposal can be sensible while authority is denied. A policy decision can allow an action while confirmation is still missing. A confirmed action can be revalidated and stopped because the order changed.

The operator view should preserve those transitions rather than showing only the final assistant message.

A concrete investigation

Imagine an operator receives a complaint that a refund "did not happen." The chat transcript contains a confident assistant response, but that is not enough to determine the outcome.

The operator should be able to answer the questions in order:

1. What did the customer request, and which workflow did the server create?
2. Which customer and target were resolved after authentication and scope checks?
3. Did policy allow the action, require confirmation, or deny it?
4. Was the confirmation bound to this exact pending action?
5. Was execution attempted, skipped, or stopped during revalidation?
6. If the result was uncertain, did replay or idempotency reconcile the outcome?

Consider three outcomes that could all produce a similar chat response:

Decision     require_confirmation    Authority    not_granted    Execution    not_attempted
Decision     allow                   Authority    granted        Execution    blocked by revalidation
Decision     allow                   Authority    granted        Execution    committed, receipt returned

These are operationally different. The first needs a customer confirmation. The second needs an explanation of the stale business state. The third needs a receipt and the resulting business projection. A single "refund failed" label loses the distinction.

Projections should preserve causality

The operator console does not need to display every internal object. It does need a bounded projection that preserves the causal chain between request, decision, authority, and effect.

A useful projection can include:

request_id
workflow_id
pending_action_id
semantic proposal
resolved target and scope status
validation and policy outcome
confirmation state
authority outcome
execution status
idempotency or replay result
RAG evidence and grounding status
trace context

The exact field names are less important than the relationships. A policy decision without its pending action is difficult to interpret. An execution result without its authority state is difficult to audit. RAG evidence without source and grounding status is difficult to trust.

The projection should also distinguish absent data from a negative result. "No execution record exists" is not necessarily the same as "execution was not attempted." One may indicate an instrumentation gap; the other may be the correct result of a confirmation boundary.

Observability is not a transcript archive

A transcript tells an operator what the customer and assistant said. It does not reliably tell them what the server accepted, rejected, or committed.

That is why raw model output should be treated as one event in a larger lifecycle. The useful record is the transition around it: the proposal was parsed, the target was resolved or rejected, policy produced a result, confirmation was bound or missing, revalidation passed or failed, and execution either did or did not happen.

This also makes privacy boundaries clearer. The console can expose enough evidence to investigate a decision without exposing hidden reasoning, secrets, raw prompts, or unnecessary customer data. Bounded projections are more useful than indiscriminate logging because they are designed around the questions an operator must answer.

What an operator needs to inspect

When a consequential request is investigated, the useful timeline is closer to:

request
  ↓
workflow lifecycle
  ↓
semantic proposal
  ↓
target and validation result
  ↓
policy and confirmation state
  ↓
authority outcome
  ↓
execution attempt or explicit non-attempt
  ↓
replay/idempotency result

For a knowledge question, the timeline should also show retrieval evidence, provenance, citation coverage, and grounding status. That evidence explains the answer. It does not explain away a missing execution permission.

Trace context connects these records across the request, workflow, retrieval, policy, and business layers. Without that connection, an operator has to infer causality from separate logs and model transcripts.

Evidence without hidden reasoning

Useful observability does not require exposing private chain-of-thought or every raw prompt. A bounded operator projection can show the semantic proposal, the selected policy outcome, the authority state, workflow transitions, execution outcome, retrieval counts, citation status, replay result, and trace identifiers.

It should avoid leaking secrets, raw tool arguments, sensitive customer data, or hidden reasoning. The aim is not to replay the model's private thought process. It is to make the server-owned decision path inspectable.

That distinction matters during an incident. "The model asked for a refund" is an input to the investigation. "The target was resolved in customer scope, policy required confirmation, authority was not granted, and execution was not attempted" is an operational explanation.

The console is a control-plane evidence surface

An operator console is valuable when it answers questions that a chat transcript cannot:

• Which workflow is active, suspended, or superseded?
• Which target and scope were validated?
• What policy result was produced?
• Was confirmation bound to this exact pending action?
• Was execution attempted?
• If it was replayed, did idempotency return an existing outcome?
• Which evidence supported a grounded answer?

The current platform uses bounded projections for these questions. The real-LLM QA report describes operator-visible workflow, decision, authority, execution, RAG, and security outcomes without exposing hidden reasoning or secrets.

From incident evidence to action

Good observability should shorten the path from an alert to a safe next step. If a workflow is suspended, the operator should see what interrupted it and whether the customer can resume it. If a workflow was superseded, the old action should be visibly inactive. If an execution attempt has an unknown outcome, the console should direct the investigation toward reconciliation rather than another blind write.

The same evidence helps during evaluation. A warning from a real model can be inspected alongside the workflow state, policy result, retrieval evidence, and execution outcome. This makes it possible to tell whether the warning was a language-quality issue or a control-plane issue without reading a long transcript and guessing.

That is the difference between observability as a dashboard and observability as a control-plane surface. The dashboard shows activity. The projection explains what the system allowed, what it refused, and what effect, if any, reached business state.

The Agentic Customer Service Platform case study shows the decision, authority, and execution separation in its operator observability section.

For consequential agents, observability is part of the safety model. If decision, authority, and execution cannot be inspected separately, a successful-looking response can hide a denied action, a stale workflow, or an unknown write outcome.