The first 60 minutes of an AI security incident

An AI incident does not begin and end with a suspicious response. The systems around the model—identities, tools, retrieval pipelines, memory, queues, and downstream applications—determine the real blast radius.

The first hour is not about proving root cause. It is about stopping additional autonomous actions, removing unsafe authority, preserving the decision trail, and establishing a defensible scope.

When unexpected behavior becomes an incident

Not every hallucination is a security incident. Escalate when the agent has crossed, or attempted to cross, a security or authorization boundary.

• Executing a tool without the expected approval.
• Accessing data outside the user's tenant, role, case, or assigned scope.
• Disclosing sensitive retrieval results in a prompt, response, trace, or external message.
• Using a delegated credential for an unintended operation.
• Writing poisoned or unauthorized content into long-term memory.
• Modifying cloud resources, identities, repositories, tickets, or production systems.
• Spawning subagents, workflows, or tool calls that responders cannot account for.
• Continuing to operate after the initiating user session has ended.

The incident threshold should be based on what the system attempted and what authority it possessed—not only on the final text shown to the user. Current NIST guidance treats incident response as part of broader cybersecurity risk management rather than an isolated forensic activity. NIST SP 800-61 Rev. 3 ↗

The first-hour operating timeline

1. 0–5 minutes: recognize and declare

Start with an authoritative signal: a cloud audit event, successful tool response, identity event, data-access record, repository commit, delivered message, policy-engine denial, or retrieval event outside the expected boundary.

A chat transcript can support the investigation, but it is not authoritative proof. It may omit retries, hidden tool calls, system instructions, retrieval context, memory operations, or subagent activity.

Do not wait for attribution, a complete record count, or a confirmed prompt-injection path before declaring the incident.

2. 5–10 minutes: establish command

AI incidents cross control planes quickly. Assign one incident commander before teams independently change the environment.

• Incident commander: priorities, scope, decisions, and update cadence.
• Agent-platform lead: deployments, aliases, queues, sessions, and subagents.
• Identity lead: principals, tokens, secrets, and session revocation.
• Evidence lead: traces, configuration, memory, and target-system logs.
• Tool owners: cloud, code, messaging, ticketing, and business-system validation.
• Communications, privacy, and legal: engaged according to potential impact.

UTC time | responder | target | previous state | action
reason | expected effect | validation | rollback path

Freeze unapproved changes to prompts, models, guardrails, policy bundles, tools, retrieval indexes, memory, aliases, and trace configuration. Emergency changes remain possible, but the commander should authorize and record them.

3. 10–20 minutes: suspend execution

1. Stop new requests at the agent gateway or API route.
2. Pause production aliases, schedules, event triggers, and message consumers.
3. Hold queued tasks and agent-to-agent handoffs.
4. Terminate in-flight work when continued execution is riskier than the evidence it contains.
5. Disable fallback routes to alternate deployments.
6. Keep telemetry and administrative access readable.

For example, Amazon Bedrock supports pausing an agent alias with REJECT_INVOCATIONS without changing its underlying IAM policy. Amazon Bedrock deployment guidance ↗

Validate the effect: new-run count is zero, queue depth is stable, no new tool proposals or subagents appear, alternate aliases are not reachable, and logs remain available. Do not delete the agent, deployment, prompt, memory, index, traces, or policy configuration.

4. 20–30 minutes: revoke authority and contain tools

Pausing orchestration does not invalidate previously issued credentials or downstream sessions. Inventory delegated OAuth tokens, workload identities, service principals, managed identities, API keys, certificates, cloud role sessions, tool-specific cookies, and downstream credentials returned by tools.

1. Block new authentication for the agent principal.
2. Revoke refresh and delegated tokens.
3. Disable affected workload identities or service principals.
4. Revoke application-managed sessions.
5. Disable exposed keys, secrets, and certificates.
6. Remove high-impact roles, grants, and OAuth consent.
7. Rotate credentials whose confidentiality cannot be established.

Identity-provider revocation is not always the end of a session. Microsoft documents that access tokens can remain usable until expiration and that applications must revoke the session tokens they issue. Validate revocation at both layers. Microsoft Entra emergency revocation guidance ↗

The tool gateway is usually the strongest enforcement point: it can stop an action even if the model, prompt, or runtime remains compromised.

5. 30–45 minutes: preserve the decision trail

Collect the agent version and alias; effective prompts and hashes; model and inference configuration; request, trace, span, and session IDs; retrieval queries, filters, results, source versions, and ACL decisions; memory reads and writes; guardrail and policy inputs and versions; tool arguments, approvals, results, retries, and handoffs; identity claims and token identifiers; queue state; target-system audit logs; clock status; and every responder action.

Google describes an agent trace as a request timeline composed of spans for operations such as LLM interactions and function calls. That structure is useful for reconstruction, but its content must be treated as sensitive evidence. Google Cloud agent tracing ↗

Collect in order of volatility

1. In-flight tasks, queues, and session state.
2. Ephemeral traces, gateway events, and temporary tool results.
3. Provider and application session records.
4. Memory and retrieval state.
5. Agent configuration, policies, and prompt versions.
6. Durable identity, cloud, and target-system audit logs.

For each export, retain timestamps and timezone, calculate a hash, record collector and source, preserve access controls, and avoid unnecessarily duplicating secrets or regulated data. NIST recommends integrating forensic collection with incident response across application, operating-system, file, and network sources. NIST SP 800-86 ↗

6. 30–60 minutes: hunt downstream

This work runs in parallel with evidence preservation. Start with what the agent could reach, not only what the transcript says it reached.

For every action, find the corresponding resource change, audit event, message ID, commit, transaction, or object version. Hunt for new identities and grants, created secrets, data reads and exports, cloud changes, commits and workflow runs, delivered or queued messages, webhooks, memory writes, vector updates, subagents, asynchronous jobs, and credentials copied downstream.

7. 45–60 minutes: communicate facts

Incident:
Detection time:
Confirmed facts:
Affected boundary:
Current containment:
Confirmed and potential impact:
Material unknowns:
Evidence preserved:
Decisions or assistance required:
Next update:
Incident commander:

Avoid declaring attribution, breach status, affected-record counts, or root cause before validation and authorization. Keep a technical channel, decision log, and stakeholder update separate. Use an out-of-band channel when the organization's messaging platform is in scope.

8. The 60-minute exit criteria

• New runs are stopped or bounded.
• In-flight and queued work is accounted for.
• High-risk identities, credentials, sessions, and tools are denied or constrained.
• Telemetry remains accessible.
• Effective prompts, policies, retrieval context, and memory are preserved.
• Agent identity and tool reachability are mapped.
• Downstream actions are attempted, successful, failed, or unknown.
• Target-system logs—not only transcripts—are being collected.
• Command, action tracking, and update cadence are active.
• Business, privacy, legal, and communications owners are engaged as required.
• The next decision and update time are recorded.

Closing position

The first hour of an AI security incident is an authority problem before it is a model problem.

Pause execution without erasing state. Revoke authority without blinding telemetry. Preserve effective prompts, policies, retrieval context, and memory—not only the visible conversation. Hunt in the systems the agent could change, and communicate what the evidence proves rather than what the team suspects.