When your AI evidence cannot be trusted

A transcript is a user interface artifact. It is not a complete forensic record of what an AI system received, decided, retrieved, remembered, or executed.

When an agent incident begins, responders need to establish which records describe actual system activity, which are incomplete observations, and which are merely model-generated claims.

Operating principle: Preserve the execution path, not only the conversation. Every important AI action should be independently verifiable outside the model.

The evidence problem

Traditional application investigations already require correlation across endpoints, identities, networks, applications, and cloud control planes. Agentic systems add several less stable layers:

• Dynamically assembled prompts.
• Model and provider configuration.
• Retrieved context.
• Guardrail and policy decisions.
• Tool proposals and arguments.
• Human approvals.
• Tool execution and results.
• Agent-to-agent handoffs.
• Short-term and persistent memory.
• Model-generated summaries of earlier activity.

Each layer can record a different version of the same event.

An assistant might say, “I did not send the message,” while the messaging platform contains a delivery receipt. A trace might record a proposed cloud action but omit whether the target accepted it. A current memory record may no longer contain the value used during the incident. A prompt stored in source control may differ from the effective prompt assembled at runtime.

The investigation therefore needs provenance: evidence of where each record originated, how it moved, whether it changed, and which independent system can confirm it.

An evidence-provenance architecture

U --> C
R --> C
G --> C
K --> C
M --> C
T --> C
I --> C
D --> C
C --> V --> H
H --> S1
H --> S2
S1 --> S3
S1 --> X
S2 --> X
S3 --> X
X --> Q --> A

The diagram deliberately separates raw evidence from operational security telemetry. A SIEM does not need unrestricted prompt contents to detect suspicious behavior. Investigators may need selected content, but that evidence belongs in a more restricted store with access logging, retention controls, and chain-of-custody records.

1. Treat the transcript as a partial observation

A visible transcript usually contains the user messages and final assistant responses. Depending on the implementation, it may omit:

• System and developer instructions.
• Prompt templates and runtime substitutions.
• Content removed by context-window truncation.
• Retrieval queries and rejected results.
• Tool calls hidden from the user interface.
• Failed calls and automatic retries.
• Guardrail decisions.
• Agent handoffs.
• Memory reads and writes.
• Provider-side preprocessing.
• Responses generated but not delivered.
• Streaming content suppressed by the application.
• Actions completed after the conversation ended.

The transcript is still useful. It establishes what the user saw and may provide timestamps, intent, or disclosure evidence. It simply cannot prove the complete execution path.

Label transcript evidence accurately:

A model statement such as “the file was deleted” is an investigative lead. The storage system’s audit event and resulting object state are evidence of whether deletion occurred.

2. Correlate tool calls through execution

A defensible tool record needs to connect four separate moments:

1. The model proposed an action.
2. The enforcement layer evaluated it.
3. The tool adapter attempted execution.
4. The target system accepted, rejected, or partially completed it.

Use a stable call identifier across those moments.

{
  "trace_id": "trace_01J...",
  "run_id": "run_01J...",
  "turn_id": "turn_009",
  "tool_call_id": "call_7ab2",
  "agent_id": "incident-agent",
  "agent_version": "2026.09.2",
  "tool": "identity_admin",
  "operation": "revoke_sessions",
  "arguments_digest": "sha256:...",
  "policy_decision_id": "decision_81fd",
  "approval_id": "approval_1092",
  "attempted_at": "2026-09-05T18:42:11.382Z",
  "target_event_id": "entra:audit:...",
  "result": "succeeded"
}

The argument digest should be calculated over a deterministic, canonical representation of the tool name, operation, target, and arguments. If the arguments change after approval, the digest changes and the approval no longer matches.

OpenAI’s Agents SDK uses traces and spans to represent generations, tool calls, handoffs, guardrails, and custom events. It also requires tool-call identifiers to remain associated with their canonical invocations and outputs. OpenAI Agents SDK tracing and model interface

Amazon Bedrock traces can include orchestration inputs, action-group invocations, knowledge-base queries, observations, failures, agent versions, sessions, and collaborator chains. However, trace collection must be enabled for the invocation. Amazon Bedrock agent tracing

Neither trace should replace the target system’s audit log. The runtime can prove what it attempted. The target system is usually better positioned to prove what changed.

3. Preserve memory as versioned state

Agent memory can be overwritten, summarized, merged, expired, or deleted before responders inspect it.

A current memory record does not necessarily reveal:

• The value read during the incident.
• Which source created the value.
• Whether a model-generated summary replaced raw content.
• Which user, tenant, agent, or session owned it.
• Whether an attacker modified it before execution.
• Whether another workflow consumed it.
• Which version influenced a tool decision.

For security-relevant memory, use append-only change events or versioned snapshots.

Each memory event should record:

• Memory record and version identifiers.
• Tenant, user, session, and agent scope.
• Creating identity.
• Source event or content identifiers.
• Previous and new content digests.
• Read, write, update, summarize, merge, or delete operation.
• Creation and expiration timestamps.
• Reason for the change.
• Policy decision authorizing the operation.
• Trace and span identifiers.
• The model or deterministic process that produced a summary.

Do not silently replace raw memory with a model-generated summary. Preserve the relationship:

raw events → source versions → summarization request
           → model/configuration → generated summary → active memory version

During an incident, disable further writes or snapshot the affected memory partition before inspecting it. Otherwise, investigative activity or continued agent execution may alter the state being investigated.

4. Capture the effective prompt

A prompt template in Git does not prove what the model received.

The effective prompt may combine:

• A version-controlled system template.
• Deployment-specific instructions.
• Feature flags.
• Tenant configuration.
• User input.
• Conversation history.
• Retrieved documents.
• Memory.
• Tool definitions.
• Policy obligations.
• Provider-added formatting.
• Previous tool results.
• Dynamic date, locale, identity, or environment values.

Preserve both the components and their final assembly order.

At minimum, record:

template ID and commit
deployment and agent version
component identifiers and versions
assembly order
runtime substitutions
tool-definition digest
retrieval-result identifiers
memory-version identifiers
model and provider
inference configuration
effective-input digest
provider request ID

Where policy permits, place the exact effective prompt in restricted evidence storage. If full content cannot be retained, preserve component identifiers, cryptographic digests, assembly metadata, classification, and a documented reason for omission.

Prompt hashes alone are not sufficient if investigators cannot reconstruct the hashed input. The hash establishes whether two available objects match; it does not recreate an object that was never preserved.

5. Retrieval context is part of the incident

An agent response may appear inexplicable because the evidence contains the generated answer but not the content supplied by retrieval.

For each retrieval operation, preserve:

• Authenticated user and workload identity.
• Tenant and authorization context.
• Query text or protected query digest.
• Query rewrite and expansion.
• Knowledge base, index, collection, and namespace.
• Index snapshot or revision.
• Embedding model and version.
• Ranking and reranking configuration.
• Security and metadata filters.
• Retrieved chunk identifiers.
• Source-object identifiers and versions.
• Chunk content digests.
• Similarity or ranking scores.
• Results rejected by authorization or policy.
• Retrieval timestamp.
• Trace, span, and parent operation.
• Content actually placed into the prompt.

The distinction between “retrieved” and “used” matters. A retriever may return ten results, an authorization filter may remove three, and the prompt builder may select four of the remaining seven. Investigators should be able to identify each transition.

Preserve source versions as well as chunk versions. A current source document may have been corrected after the incident, while the vector store still reflects an earlier version.

6. Detect prompt-version drift

Prompt-version drift occurs when the effective runtime behavior no longer matches the reviewed or approved configuration.

Common causes include:

• A prompt was edited directly in a provider console.
• A deployment alias points to an unexpected version.
• An environment variable overrides repository configuration.
• A stale container or worker remains in service.
• A feature flag changes the prompt assembly path.
• A tool schema changes without a corresponding prompt revision.
• A retrieval template or policy bundle updates independently.
• A rollback restores only part of the agent stack.

Record a deployment manifest that binds together:

application commit
agent version
prompt-template digest
tool-schema digest
policy-bundle digest
retrieval configuration
memory schema
model identifier
provider configuration
release timestamp
approver

At runtime, emit that manifest identifier into each trace. Then alert when active telemetry reports a combination that does not correspond to an approved manifest.

7. Timestamp integrity is an architectural control

A timestamp is not automatically trustworthy because it ends in Z.

Agent evidence can cross:

• Browser clocks.
• Application servers.
• Serverless functions.
• Model providers.
• Tool gateways.
• Identity providers.
• SaaS platforms.
• Cloud control planes.
• Queues and asynchronous workers.
• Regional log pipelines.

Each system may have different clock sources, precision, buffering, and ingestion delays.

For important events, retain:

• The source-generated event time.
• Collection or ingestion time.
• Start and end time.
• Timezone and UTC offset.
• Timestamp precision.
• Clock source.
• Measured clock skew.
• Monotonic sequence number when available.
• Queue enqueue and dequeue times.
• Provider and target-system event identifiers.

NIST SP 800-53 includes controls for timestamps that map to UTC and synchronization with authoritative time sources, as well as protection of audit information. NIST SP 800-53 Rev. 5

Do not rewrite original provider timestamps during normalization. Store normalized UTC alongside the original value and document the transformation.

When clocks disagree, use causal relationships to constrain the sequence:

request accepted
  → tool proposed
    → policy evaluated
      → queue message created
        → target event recorded
          → result consumed

Report uncertain time as a range rather than manufacturing false precision.

8. Understand provider logging limitations

Provider traces can be extremely useful, but availability and content depend on the service, SDK, configuration, data controls, and retention policy.

Examples:

• Amazon Bedrock runtime tracing is optional and can include prompts, orchestration, action-group inputs, knowledge-base queries, observations, and failures.
• OpenAI Agents SDK tracing can record generations, tool calls, handoffs, and guardrails, but it can be disabled. Its documentation also notes that tracing is unavailable under Zero Data Retention and that generation and function spans can contain sensitive inputs and outputs.
• Microsoft Foundry traces may contain prompts, responses, system instructions, tool definitions, tool arguments, and tool results. Microsoft provides a mechanism for separating this sensitive content into a protected table with distinct access control.
• OpenTelemetry’s GenAI attributes provide useful correlation conventions, but several content-bearing attributes are sensitive, verbose, or optional. Instrumentation and semantic-convention versions may also differ.

Sources: OpenAI Agents SDK tracing, Microsoft Foundry sensitive trace controls, and OpenTelemetry GenAI attributes.

Document for each provider:

“Tracing enabled” is not a sufficient answer.

9. Build tamper-evident evidence collection

Centralized logs help responders correlate activity, but the evidence path also needs protection from unauthorized modification or deletion. CISA recommends centralizing logs, restricting access, storing them securely, and retaining them according to policy. CISA logging guidance

For high-value AI events:

1. Generate a unique event identifier at the source.
2. Add trace, span, session, run, turn, and tool-call identifiers.
3. Validate the event against a versioned schema.
4. Canonicalize the security-relevant fields.
5. Calculate a content digest.
6. Link batches or events using a previous digest where appropriate.
7. Send raw evidence to restricted storage.
8. Send a redacted operational copy to detection systems.
9. Periodically sign an export manifest.
10. Store incident exports under immutable retention.
11. Log every read, export, retention change, and deletion attempt.

A simple evidence envelope might contain:

{
  "schema_version": "ai-evidence/1.0",
  "event_id": "evt_01J...",
  "event_type": "agent.tool.result",
  "source_system": "tool-gateway-prod",
  "source_event_id": "gateway:918221",
  "occurred_at": "2026-09-05T18:42:11.491Z",
  "ingested_at": "2026-09-05T18:42:11.842Z",
  "sequence": 184,
  "trace_id": "trace_01J...",
  "tool_call_id": "call_7ab2",
  "actor_id": "workload:incident-agent",
  "tenant_id": "tenant-blue",
  "payload_digest": "sha256:...",
  "previous_event_digest": "sha256:...",
  "collector_version": "collector-3.4.1",
  "redaction_profile": "security-standard-2"
}

Cryptographic digests provide integrity evidence only if the trusted digest or signed manifest is stored separately from the data an attacker could alter.

10. Minimum viable forensic record

NIST’s forensic guidance recommends collecting and analyzing evidence across multiple data sources rather than treating one application record as the entire investigation. NIST SP 800-86

11. Reconstruct honestly when evidence is missing

Missing evidence should not be silently replaced by inference.

Use explicit confidence labels:

• Confirmed: supported by an authoritative system-of-record event or multiple consistent independent sources.
• Corroborated: supported by two or more sources, though no single source proves the event completely.
• Reported: asserted by a user, model, tool response, or unverified record.
• Inferred: logically derived from surrounding events and stated assumptions.
• Unknown: evidence is absent, contradictory, or outside retention.
• Disproved: authoritative evidence contradicts the claim.

For every reconstructed event, record:

claim
supporting sources
contradicting sources
missing evidence
assumptions
confidence
analyst
assessment time

Do not edit the original timeline when confidence changes. Create a new assessment version and retain the previous one.

12. Test whether the evidence can answer an incident

Run exercises that force the telemetry to prove:

• Which effective prompt the model received.
• Which retrieved chunks entered the context.
• Which memory version influenced the decision.
• Which policy revision allowed or denied the action.
• Whether approval matched the executed arguments.
• Whether a tool was proposed, attempted, and completed.
• Which identity the target system observed.
• Whether a retry caused a duplicate action.
• Whether another agent inherited the task.
• Whether an asynchronous job continued after containment.
• Whether an administrator modified or deleted evidence.
• Whether regional clocks and queues can be reconciled.
• Whether sensitive traces are properly restricted.
• Whether investigators can export the evidence before retention expires.

Include failure cases:

• Provider tracing disabled.
• Trace sampling drops the incident run.
• Tool gateway unavailable.
• Memory overwritten.
• Prompt changed directly in a console.
• Retrieval index rebuilt after the incident.
• Target audit event delayed.
• Clock skew exceeds the accepted threshold.
• Sensitive content captured in a broadly accessible trace.
• An attacker has administrative access to the logging platform.

An evidence design is ready only when responders can reconstruct the incident despite one missing or compromised telemetry source.

Field checklist

• Transcripts are classified as partial observations.
• Agent runs have stable trace, session, turn, and call identifiers.
• Tool proposals correlate with policy, approval, execution, and target events.
• Effective prompts are reconstructable.
• Prompt, tool, policy, retrieval, and deployment versions are bound together.
• Retrieval evidence includes source versions and authorization decisions.
• Memory is versioned or change events are append-only.
• Source and ingestion timestamps are preserved separately.
• Clock skew and sequence gaps are detectable.
• Provider trace coverage and exclusions are documented.
• Sensitive trace content has stronger access controls than general telemetry.
• Raw evidence and redacted detection telemetry are separated.
• Evidence exports have hashes, manifests, retention, and custody records.
• Investigative conclusions carry explicit confidence.
• Exercises test missing, delayed, modified, and contradictory evidence.

Closing position

An AI system can produce a convincing narrative about what it did. That does not make the narrative evidence.

A defensible investigation connects the user request to the effective prompt, retrieved context, memory state, policy decision, approval, tool execution, target-system event, and final response. It preserves the identifiers and versions that make those records comparable. It also states clearly when a link is missing.

The boundary: The model may explain an action. Only independently preserved and correlated records can establish that the action occurred.