MCP server security incident response playbook — what to do in the first 60 minutes

MCP server incidents differ from conventional web application incidents in one critical way: the blast radius is determined by ambient authority, not by session scope. A compromised web application can only act within the permissions granted to the authenticated session. A compromised MCP server acts with the full capability set of its long-lived credentials — the Stripe key, the database connection string, the GitHub personal access token — regardless of what the LLM was originally asked to do. The attacker does not need to escalate privileges. The privileges were already there.

The second difference is speed. A human attacker probing a web application acts at human speed. An LLM orchestrator operating on attacker-injected instructions acts at machine speed: hundreds of tool calls per minute, across every tool in the server, before any human notices the anomalous traffic. By the time the on-call alert fires, the exfiltration loop may have already completed.

These two properties — ambient credentials and machine-speed exploitation — mean that the response window for an MCP server incident is measured in minutes, not hours. The playbook below is designed accordingly.

How you know you have an incident

Confirmed incidents usually arrive via one of four channels. The detection channel affects how much context you have at the start of the response, and therefore how you prioritize the first ten minutes.

60-minute timeline overview

Your first job is not to act — it is to understand. Confirm that you are looking at actual exploitation, not a false positive, a misconfigured alert, or a legitimate but unusual usage pattern. Then identify what you are dealing with before you start changing anything.

• Pull the last 500 lines of your audit log. Look for tool calls that are out of sequence, out of hours, or that access data outside the normal usage pattern. A bulk enumeration loop (sequential IDs, repeated calls to a list-type tool) is the most common pattern.
• Identify the credential set the server holds: database connection strings, third-party API keys, object storage credentials, OAuth tokens. Write this list down now — you will need it in Phase 3.
• Identify the entry point if visible: a specific tool handler, a specific session ID, a specific source IP. The audit log should contain this if it was implemented correctly.
• Check whether the server is still running and actively processing calls. If yes, the incident is active and you proceed to Phase 2 immediately. If the server is already down (crash, pod restart, process kill), you have more time for scoping before isolation.

The audit log is your primary forensic tool. If you implemented the append-only audit log pattern described in our architecture guide — every tool call, its arguments hash, and its result — you will have a complete record of what the server did. If you did not implement an audit log, the scoping phase will be significantly harder: you are working from downstream service logs, and reconstruction is slow.

Isolation stops the attack from continuing. Do not attempt to understand the full scope before isolating — that sequence is backwards. Isolation first, then triage. An active attack that you are analyzing while it continues is extending its own blast radius during your analysis time.

• Pull the server process. pm2 stop mcp-server, docker stop <container>, systemctl stop mcp-server.service, or equivalent. The goal is to stop tool handlers from executing new calls.
• If the server is behind a load balancer or reverse proxy, remove it from the upstream pool. This allows the process to remain running for memory forensics while preventing new traffic from reaching it.
• Revoke any active OAuth tokens or session tokens your server issued. These are separate from the long-lived credentials rotated in Phase 3 — they are the tokens that sessions may be actively using to keep calling the now-isolated server via a proxy or cached client.
• If the server is containerized: docker network disconnect <network> <container> to cut network access while keeping the container running for forensics.
• Block the source IP at the network layer if one was identified in Phase 1. This is a low-cost containment step that costs nothing if the IP rotates and buys time if it does not.

Credential rotation is the most consequential action in the playbook. Done correctly, it neutralizes exfiltrated credentials before they can be used. Done wrong, it causes cascading failures in unaffected downstream systems. Work from the list you wrote in Phase 1.

• Database credentials first. Your database connection string is usually the highest-value credential: it provides access to all application data. Rotate it first. For PostgreSQL: create a new role, update the application's secret, revoke the old role. Do not just change the password on the existing role — if the attacker has the current credentials, they can observe the rotation and reconnect before the old password is deactivated.
• Third-party API keys. For each service in your credential list (Stripe, OpenAI/Anthropic, GitHub, AWS, etc.): revoke the old key immediately at the vendor's dashboard, then provision a replacement. Most vendor dashboards support atomic key rotation — provision new, update secret store, revoke old — without downtime.
• Do not rotate credentials for systems the server did not have access to. Unnecessary rotation of unaffected systems creates outage risk for no security gain. Your Phase 1 credential list is the scope boundary.
• Update your secret store. Whether you use AWS Secrets Manager, Doppler, Vault, a .env file, or environment variables injected at deploy time — update the store before attempting to restart the server. A server restarted with the old credentials is immediately re-compromised if the credentials were not yet revoked.
• Notify vendors if credentials were exposed for more than 30 minutes. Stripe, Anthropic, and GitHub all have trust and safety teams that can audit usage of a compromised key and provide a log of what was called. This is free forensic intelligence and you should use it.

By minute 30, the attack is contained and credentials are rotated. The incident is no longer spreading. Now preserve the forensic record before anything overwrites it.

• Export the full audit log to a write-once destination (S3 with Object Lock, a timestamped archive on a different host, a local file with permissions changed to read-only). Do this now, before log rotation cycles the file.
• Compute and record the SHA-256 hash of the exported audit log. sha256sum audit.log > audit.log.sha256. This creates a cryptographic record of the log state at the time of export that cannot be falsified after the fact.
• Export downstream service logs for the incident window: database query logs for the relevant time range, object storage access logs, API gateway logs. These corroborate the audit log and fill gaps if tool call arguments were not fully logged.
• Take a filesystem snapshot of the server's working directory: tar czf server-snapshot-$(date +%Y%m%dT%H%M%S).tar.gz /path/to/server. This preserves the code, config, and any files the attacker may have written.
• If the server is containerized, commit the container image: docker commit <container> incident-snapshot-$(date +%Y%m%d). This preserves the full OS state including any modifications the attacker made to the container filesystem.

Triage answers three questions that drive every decision downstream: What happened? What data was accessed or exfiltrated? What was the entry point?

What happened

Walk the audit log from the first anomalous entry backward to find the initial compromise point, and forward to identify the full action sequence. The most common MCP server incident patterns are:

What data was accessed

This question drives your breach notification obligation. Cross-reference the audit log tool call arguments against your data model:

• For reads: what record IDs were accessed? Map IDs to data classes (PII, financial data, health data, authentication data). If the tool accepts arbitrary IDs and a bulk enumeration occurred, assume all accessible records were accessed.
• For writes: what values were written? Were any records deleted or modified? Identify the scope of data integrity impact.
• For external calls: what destination hosts were contacted? What was sent? If your audit log hashes arguments rather than logging them in full (the correct implementation for credentials), you cannot reconstruct the exact data sent — but you can reconstruct the data type from the tool definition.

What was the entry point

Identifying the entry point is required for root cause analysis (Phase 6, post-incident) but also for the communication phase: you need to know whether the vulnerability is patched before you bring the server back online. If the entry point is an unpatched vulnerability, do not restart the server until the fix is deployed.

By minute 55, the incident is contained, credentials are rotated, evidence is preserved, and you have a triage summary. Now communicate — in the right order.

Internal stakeholders (minute 55)

Send a brief status to your team: what happened, what you did, what the current status is (contained / not contained), and what the immediate next steps are. Do not wait for a complete post-mortem to do this — a summary message is more valuable now than a thorough one in two hours.

Vendors (minute 56)

Notify each vendor whose credentials were rotated that the rotation was incident-triggered, not routine. This is not a legal obligation but a practical one: the vendor's trust and safety team can provide usage logs, flag any accounts that were provisioned with the compromised key, and add the old key to their watchlist for continued abuse detection.

External notification (if required, minute 57–60)

Whether you have a notification obligation depends on what data was accessed, your jurisdiction, and your terms of service. The triage output from Phase 5 gives you the data class, the record count, and the time window — the three inputs your legal counsel needs to determine whether GDPR, CCPA, SOC 2, or contractual notification requirements are triggered.

After the 60-minute window

The first sixty minutes end the acute phase. The post-incident phase covers root cause analysis, vulnerability remediation, return-to-service validation, and the post-mortem process. Key steps:

Root cause analysis and remediation

Identify the exact code path that was exploited. In most MCP server incidents, the root cause falls into one of the categories SkillAudit flags statically: a prompt injection vulnerability in a tool that reads external content, a missing ownership check in a data access tool, an unsafe credential handling pattern that made exfiltration possible, or an over-declared permission set that increased the blast radius of what would otherwise have been a low-impact exploit.

Run a fresh SkillAudit scan on the server repository after applying the fix. Compare the result to your pre-incident scan. If the finding that corresponds to the entry point is gone, the remediation is complete. If it persists, the fix is incomplete — do not restart the server.

Return-to-service validation

• Deploy the patched server with the new credentials provisioned in Phase 3.
• Run a SkillAudit scan on the deployed version before routing any traffic to it.
• Confirm the audit log is recording correctly by executing a known-good tool call and verifying the log entry.
• Re-enable traffic incrementally — route a fraction of traffic through the patched server, observe for anomalies, then complete the cutover.

The post-mortem

A post-mortem serves two functions: it creates institutional memory (so the same class of vulnerability does not recur), and it creates documentation for any external party who needs to verify that you addressed the incident responsibly. The post-mortem should cover the timeline, the root cause, what controls failed (or were absent), what controls worked, and what changes were made.

The most useful post-mortems are specific. "We hardened our input validation" is not actionable. "We added Zod schema validation to all tool argument inputs and enabled strict mode to reject extra keys" is actionable and verifiable.

Using SkillAudit as a continuous incident readiness tool

The best incident response is the incident you detect before it becomes an incident. SkillAudit's continuous scan history gives you three capabilities that make detection earlier and response faster:

Pre-incident baseline. A SkillAudit report run before an incident gives you a documented snapshot of the server's security posture at a known-good point in time. During triage, comparing the current server state to the pre-incident baseline immediately surfaces any modifications the attacker made to the server's code or configuration.

Grade drift detection. SkillAudit's history view tracks your score across the six audit axes over time. A declining score in the Credential Exposure or Permissions Hygiene axis — even before a confirmed incident — is an early warning that the server's risk surface is growing. Addressing score regressions proactively is cheaper than responding to the incident they predict.

Post-incident verification. After remediation, the SkillAudit report is the auditable record that the vulnerability was addressed. The pre-incident report shows the finding; the post-incident report shows it absent. This record is directly useful for any external party — a user, a regulator, a platform that lists your server — who needs to verify that you responded to the incident responsibly.

For servers in the Pro tier, the CI webhook integration creates a continuous gate: every push to the server repository triggers a SkillAudit scan, and a grade below your configured threshold blocks the deployment. This does not prevent zero-day exploitation, but it ensures that any vulnerability that SkillAudit can detect statically cannot reach production in the first place — which covers the majority of the incident categories described above.

Incident response readiness checklist

These are the controls that determine how fast and how cleanly you can respond to an incident. They are worth implementing before you need them.

• Append-only audit log implemented in every tool handler, recording call timestamp, tool name, argument hash, result summary, and session ID. Stored to a write-once destination separate from the server process.
• Credential inventory documented: every long-lived credential the server holds, the vendor, the scope granted, and the rotation procedure. This list should be accessible without running the server.
• Zero-downtime credential rotation validated: you have tested rotating each credential without a service outage at least once, before an incident forces you to do it under pressure for the first time.
• Downstream service logging enabled: database query logs, object storage access logs, and API gateway logs retained for at least 30 days.
• SkillAudit scan scheduled on every release, with results retained in scan history for baseline comparison.
• Incident response contact list written down: vendor trust and safety contacts, legal counsel for notification questions, internal stakeholder distribution list.
• Server isolation procedure tested: you know exactly which command pulls the server from production traffic without a database transaction being left open or a credential being written to disk.

Related reading

• Minimal-footprint MCP server architecture — the structural patterns that prevent most incidents from being exploitable in the first place
• SkillAudit grade drift analysis — how to detect rising risk before it becomes an incident, using score trajectory
• Anatomy of a prompt injection attack on an MCP server — the most common MCP server incident entry point, dissected step by step
• MCP server security checklist — pre-ship controls that reduce the probability of the incidents this playbook responds to
• GitHub Action MCP security gate — CI integration for continuous pre-production scanning
• MCP server audit logging implementation — technical patterns for the append-only audit log that this playbook depends on
• MCP server API key rotation security — zero-downtime credential rotation patterns for Phase 3 of this playbook