AI Agent Observability: The Evidence Layer

AI agent observability: the fast answer

That is the practical standard. Not a colorful dashboard. Not a list of token counts. Not a generic log stream. The system should let an operator answer five questions quickly: What was the agent asked to do? What evidence did it use? What did it decide and execute? What actually changed? Was the business outcome correct?

Traditional application monitoring answers whether software is available, fast, and error free. AI agent observability must also answer whether the work was appropriate, grounded, authorized, complete, and useful. That requires technical telemetry and business evidence in the same trace.

Why normal logs are not enough for AI agents

A conventional application follows code paths written in advance. An agent chooses among tools, sources, models, and next steps at runtime. Two runs with similar inputs can take different paths. The interesting failure is often not an exception. It is a reasonable-looking sequence that produces the wrong business result.

Consider a service agent that receives a refund request. The model identifies the customer, retrieves an order, checks policy, calls a refund tool, updates the CRM, and drafts a confirmation. Every API can respond successfully. The run can still fail if the retrieved order belongs to a different customer, the policy document is outdated, the agent selected the wrong refund reason, the approval threshold was misapplied, or the payment processor accepted a duplicate request.

Logs may show six successful calls. Observability must show whether those calls formed a correct piece of work.

Production teams need all three. Monitoring without traces cannot reconstruct a run. Traces without evaluation only show what happened, not whether it was good. Evaluation without production telemetry cannot explain why quality changed or which component caused the regression.

This is also where observability must remain distinct from the AI agent control plane. The control plane defines and enforces identity, permissions, policies, approvals, quotas, and model routing. The observability layer records evidence, diagnoses behavior, scores outcomes, and supports replay. One controls what may happen. The other proves what did happen and whether it worked.

The seven-part agent evidence chain

An agent trace becomes useful when it follows the logic of the business task, not merely the architecture of the code. The evidence chain below links intent to outcome through seven stages.

1. Business intent

Record the task ID, requesting actor, workflow name, intended outcome, risk class, and authority boundary. “Help the customer” is not observable. “Draft a refund recommendation for order 4812, under policy version 7, without executing payment” is observable.

2. Context assembly

Record source identifiers, retrieval query, document versions, freshness, permissions, and which pieces of context were actually passed to the model. Do not depend on a full raw prompt dump. Store hashes or stable references where sensitive content should remain in its source system.

3. Model decision

Record model provider, model name, agent version, prompt or policy version, output format, sampling settings where relevant, and the decision produced. Capture enough to compare releases. Do not treat hidden chain-of-thought as a required audit artifact. The useful evidence is the explicit decision, cited context, selected action, and structured rationale designed into the workflow.

4. Tool call

Record the tool contract, call ID, argument schema version, redacted argument summary, validation result, retries, timeout, response class, and whether the tool is read-only or state-changing. A tool name alone is not enough. Operators need to know which action was requested against which entity under which limits.

5. Enterprise action

Record the system of record, before-and-after state reference, approval identity, idempotency key, transaction ID, and rollback handle. This stage separates a recommendation from a real-world change. If the agent can write, the trace must reach the write.

6. Outcome

Verify what happened after execution. Did the customer receive the refund? Did the shipment update propagate? Did the code pass deployment checks? Did the escalation reach an owner? Outcome evidence may arrive later than the agent response, so the trace model must support delayed outcome events.

7. Evaluation and replay

Attach deterministic checks, model-based scores, human review, incident labels, and replay results. A replay should run the same case against a new agent version or policy without repeating the original side effect. That requires separating decision simulation from action execution.

The agentic data layer supplies governed context and stable business meaning. The evidence chain records how that context was used. Together they prevent a common failure: an agent makes a technically valid decision from semantically wrong or stale data.

A minimum viable trace schema

Different frameworks use different names, but the portable structure is session, trace, and span. A session groups related work over time. A trace represents one end-to-end unit of work. Spans represent meaningful operations inside the trace.

The OpenAI Agents SDK tracing documentation describes traces for workflows and spans for model generations, tool calls, handoffs, guardrails, and custom events. OpenTelemetry’s GenAI semantic conventions define common attributes for agents, conversations, operations, tool calls, data sources, token use, and evaluation scores. These standards make it possible to avoid a telemetry model locked to one framework.

{
  "trace_id": "tr_01J7K8...",
  "session_id": "case_78421",
  "workflow": "customer_refund_review",
  "environment": "production",
  "agent": {
    "id": "refund_reviewer",
    "version": "2026.08.3",
    "model": "approved-model-alias",
    "policy_version": "refund-policy-7"
  },
  "intent": {
    "requester_type": "support_user",
    "outcome": "valid recommendation for the correct order",
    "action_tier": "draft_only"
  },
  "context": {
    "sources": ["crm:customer:hash", "orders:4812", "policy:refund:7"],
    "freshness_seconds": 42,
    "content_recorded": false
  },
  "spans": [
    {"name": "retrieve order", "status": "ok", "latency_ms": 118},
    {"name": "evaluate policy", "status": "ok", "latency_ms": 641},
    {"name": "draft recommendation", "status": "ok", "latency_ms": 902}
  ],
  "outcome": {
    "verified": true,
    "quality_score": 0.94,
    "human_override": false,
    "cost_usd": 0.034,
    "latency_ms": 1812
  }
}

Use low-cardinality names for spans. “execute refund” is useful for dashboards. “execute refund for customer 739184 at 14:32” is not. Put unique IDs in attributes, where they can be filtered without turning every run into a new metric series.

Propagate the trace ID beyond the agent framework. Put it in API headers, queue messages, database audit records, approval events, and downstream job metadata. Without propagation, the trace stops at the model boundary and cannot prove what happened in the enterprise system.

The AI agent evidence contract

Before choosing a dashboard, agree on the record that every consequential run must produce. This is the smallest practical handoff between the workflow owner, engineering, security, operations, and the person accountable for the business result. It makes a trace useful when an incident happens, rather than merely available.

{
  "trace_id": "unique end-to-end run ID",
  "workflow": "one named business workflow",
  "intent": "the measurable task and expected result",
  "requester_ref": "redacted user or system reference",
  "agent_version": "released agent version",
  "policy_version": "active policy or prompt contract",
  "authority_tier": "draft | approval_required | write",
  "context_refs": ["source ID and version, not raw sensitive text"],
  "target_entity_ref": "validated customer, order, account, or asset reference",
  "tool_calls": [{"tool": "name", "contract_version": "v1", "result": "ok|blocked|failed"}],
  "approval_ref": "human or policy approval reference when required",
  "system_of_record": "the system whose state matters",
  "transaction_ref": "idempotency key or transaction ID for a write",
  "outcome_proof": "independent evidence that the intended result occurred",
  "evaluation": {"entity_match": "pass|fail", "policy_check": "pass|fail", "outcome_check": "pass|fail"},
  "rollback_handle": "safe reversal, cancellation, or recovery reference",
  "retention_class": "data handling and deletion rule"
}

Download the one-page Evidence Contract (PDF)

Use the contract as a release gate

1. Named authority: the trace states whether the agent was drafting, awaiting approval, or permitted to write. A tool response does not create authority.
2. Validated target: a state-changing action carries the exact entity reference that passed deterministic validation. Never infer identity from a model sentence alone.
3. Outcome proof: the result comes from the system of record or a defined human check, not the agent’s own completion message.
4. Recovery reference: operators can identify the idempotency key, cancellation, rollback, or manual remediation path before an incident occurs.
5. Data boundary: the trace carries stable references and redacted summaries by default. Raw prompts, documents, and tool arguments require an explicit need and retention rule.

OpenTelemetry provides a portable baseline for agent, model, tool, data-source, and evaluation attributes. The business fields above are the local contract that links those technical events to accountable enterprise work. Keep the names stable as frameworks and vendors change.

The golden signals for production agents

Token use, latency, and exceptions matter, but they are not enough. Measure the agent at three levels: system health, trajectory quality, and business outcome.

The financial measurement belongs with the AI agent ROI framework. Observability supplies the trace-level facts that make ROI credible: cost by run, review time, failure cost, recovery cost, and verified outcome. Without those facts, agent ROI becomes an estimate based on activity rather than results.

Seven failures your dashboard must reveal

The existing guide to AI agent runtime controls and insider risk explains how to prevent unsafe execution. Observability closes the loop after and during execution by showing which controls fired, which failed, and which new failure pattern should become a control.

The incident replay workflow

When an agent fails, the goal is not to read every log line. The goal is to reconstruct the smallest causal chain, protect the business, convert the failure into a test, and prove the fix without repeating the side effect.

A good incident system reduces mean time to explanation, not only mean time to recovery. Recovery restores service. Explanation prevents the same class of failure from returning under a different prompt or tool.

Worked example: prevent a false success before it refunds the wrong order

This is a worked example, not a customer case study. It shows how an evidence contract changes the response to a common enterprise failure: every API call succeeds, but the business result is wrong.

A support agent receives a request to refund an order. It retrieves a customer record, selects an order, checks the policy, calls the payment processor, updates the CRM, and drafts a confirmation. The trace looks healthy. The processor returns 200 OK. But the customer reference matches two records and the agent refunds an earlier order.

Build three checks before write mode

1. Entity check: compare the customer, order, currency, and payment reference across the CRM, order system, and payment processor. Any mismatch forces approval mode.
2. Policy check: record the policy version and deterministic eligibility result. The model can summarize policy; it should not silently replace the rule engine.
3. Outcome check: do not mark the task successful until the payment processor refund ID, CRM event, and correct order reference are linked to the original trace.

When the outcome check fails

Contain the workflow first. Disable autonomous writes for that action tier, preserve the trace, and switch the workflow back to approval mode. Then use the trace to isolate the first bad link: ambiguous entity selection, stale CRM data, missing payment reference, or a faulty tool contract.

Next, sanitize the case into a regression test. The expected result is not simply “refund rejected.” It is: select no order when the identity evidence is ambiguous; request clarification or approval; do not call the payment tool; and leave every system unchanged. Replay that test against the repaired agent with the payment tool replaced by a dry-run contract. Only restore write mode after the entity, policy, and outcome checks pass together.

How to build an evaluation loop that reflects real work

Evaluation should cover the full trajectory. A polished final answer can hide a poor retrieval step, an unnecessary tool call, a policy violation, or an expensive loop. Score the decision path and the outcome separately.

Run evaluations at four moments. Before release, run the regression dataset. During rollout, compare the new version against the current version in shadow mode. In production, score a sample of live traces and all high-risk actions. After an incident, add the sanitized case to the test set and replay it before reopening autonomy.

Do not optimize a single quality score. A version can become more accurate but slower, more expensive, less willing to escalate, or more likely to call tools. Use a release scorecard that includes outcome quality, forbidden-action rate, tool-choice accuracy, human override, cost, tail latency, and recovery.

The broader agentic enterprise operating system needs memory, orchestration, governance, and evaluation. This guide provides the evidence contract that connects those layers instead of leaving each platform with isolated dashboards.

A tool-neutral production architecture

Use open trace propagation as the spine. Instrument the agent framework, but do not stop there. Export traces through an OpenTelemetry-compatible path, enrich them with business metadata, route sensitive content according to policy, and join technical traces to outcome events from enterprise systems.

Current official platforms increasingly follow this shape. Microsoft Foundry uses tracing to capture latency, exceptions, prompts, retrieval, ordered actions, and tool calls, with OpenTelemetry and Application Insights underneath. Amazon Bedrock AgentCore Observability exposes traces, workflow steps, intermediate outputs, latency, token use, errors, and OpenTelemetry-compatible data in CloudWatch.

Agent-focused observability products can add trace graphs, prompt/version comparison, datasets, evaluators, and human annotation. Langfuse’s trace guidance, for example, recommends one self-contained unit of work per trace, meaningful nesting, stable names, business metadata, environments, and sessions for related traces. The principle matters more than the vendor: the trace structure must survive a tool change.

A practical 30-day implementation plan

The day-30 exit criteria

• Every run has a trace ID that reaches the system of record.
• Every write identifies agent, requester, policy version, tool, approval, transaction, and rollback reference.
• Every production outcome can be verified independently from the agent’s own completion message.
• Five known failure classes trigger an alert, block, escalation, or explicit low score.
• A failed case can be replayed without repeating a real-world side effect.
• Operators can compare agent versions by verified outcome, override, cost, and tail latency.
• Prompt and tool content follow a documented redaction and retention policy.

Only after those criteria are met should the team expand autonomy or add more workflows. The strongest enterprise AI agent use cases are not the ones with the most impressive demos. They are the ones where the outcome can be proven and failure can be contained.

The vendor and architecture checklist

Do not buy an observability product because the trace graph looks attractive. Test whether it can support your operating model, data boundary, and incident workflow.

Run a proof with your hardest workflow, not a vendor-provided chatbot. Bring ten normal cases, ten edge cases, and five sanitized incidents. Require the vendor to reconstruct a failure, show the sensitive-data boundary, compare two releases, and calculate cost per verified outcome.

What leaders should ask in the operating review

Executives do not need to inspect span trees every week. They need an operating view that translates evidence into decisions.

The dashboard should make those questions answerable in minutes. If leadership only sees number of agents, number of conversations, token volume, and average latency, the organization is measuring activity while the real operational risk remains hidden.