Incident Postmortem Template
Blameless, factual, focused on systems and process rather than individuals. Fill this in as soon after the incident as you can, while details are still fresh.
Draft it within 24 to 48 hours and hold the review meeting within a week.
How to use this document
- One postmortem per incident, regardless of how small it feels, small incidents often reveal the same gap that will cause a big one later.
- Write it assuming the reader was not there. No inside references, no "you know how X always does that."
- Blameless means: describe what the system and process allowed to happen, not what a person did wrong. If a person made a judgment call that looks bad in hindsight, ask why it looked reasonable at the time, that's usually where the real fix is.
Summary
| Field | Value |
|---|---|
| Incident ID / link | [tracker link] |
| Date & time (start) | [YYYY-MM-DD HH:MM TZ] |
| Date & time (resolved) | [YYYY-MM-DD HH:MM TZ] |
| Duration | [Xh Ym] |
| Severity | [SEV1/2/3] |
| Author | [name] |
| Status | [Draft / Under review / Final] |
One-paragraph summary (write this last, after everything below is filled in):
[What broke, for how long, who/what was affected, and how it was fixed. Write it in plain language a non-engineer exec could read.]
Impact
- Who was affected: [all users / a specific segment / internal only]
- What broke: [specific feature/flow]
- Scale: [% of requests failed, # of users affected, revenue/SLA impact if known, use real numbers from dashboards, not estimates]
- Customer-visible? [yes/no, did they see an error, a slowdown, wrong data, silence?]
Timeline
Use the timezone your team standardizes on. Pull timestamps from logs, dashboards, or chat, not memory.
Include the boring parts too. "Nothing happened for 20 minutes because the alert was misrouted" is exactly the kind of detail that leads to a real fix.
| Time | Event |
|---|---|
| HH:MM | [Change/deploy/event that started the chain, if known] |
| HH:MM | First alert fired / first customer report received |
| HH:MM | On-call acknowledged |
| HH:MM | Incident declared / severity assigned |
| HH:MM | [Investigation step, what was checked, what was ruled out] |
| HH:MM | Mitigation applied (e.g. rollback, feature flag off, scaled up) |
| HH:MM | Confirmed recovery on dashboards |
| HH:MM | Incident closed |
Root cause
Distinguish the triggering event from the root cause. "A deploy at 14:02" is a trigger.
"Our retry logic doesn't have a backoff cap, so a slow downstream call turned into a connection pool exhaustion" is closer to root cause. If it's not obvious, run the Five Whys.
See the RCA template for a deeper structure.
[Explain the actual mechanism of failure. Include the contributing factors: there's usually more than one, e.g. a code bug, plus a missing alert, plus an undocumented assumption, plus no staging parity.]
What went well
Don't skip this. It's how you keep good practices instead of just chasing bad ones.
- [e.g. Alert fired within 90 seconds of the actual degradation]
- [e.g. Rollback took under 5 minutes because the deploy tool supports one-command revert]
What went poorly
- [e.g. No dashboard existed for this failure mode, so diagnosis took 40 minutes of manual log grepping]
- [e.g. Runbook for this alert was out of date]
Where we got lucky
The near-misses. If this had happened at 3am on a Friday instead of 2pm on a Tuesday, would the outcome have been much worse?
Say so.
- [e.g. Secondary on-call happened to be already awake and jumped in immediately]
Action items
Every item needs an owner and a due date, or it won't happen. Split them into "must do before this can recur" and "would help but isn't blocking."
| Action | Owner | Priority | Due | Status |
|---|---|---|---|---|
| Add alert for connection pool utilization >80% | [name] | P0 | [date] | [ ] |
| Cap retry backoff in payment client | [name] | P0 | [date] | [ ] |
Update runbook for HighErrorRate-checkout-api | [name] | P1 | [date] | [ ] |
| Add load test for this failure mode to CI | [name] | P2 | [date] | [ ] |
Example: filled summary
Summary: checkout API returned elevated 500s for 23 minutes on 2026-03-04, starting at 14:02 KST, affecting an estimated 6% of checkout attempts during the window. Root cause was a retry loop in the payment provider client with no backoff cap.
A brief downstream latency spike turned into full connection pool exhaustion. We mitigated it by rolling back the deploy that shipped the new client version, then fixed it properly by adding a capped exponential backoff before re-deploying two days later.
이 템플릿을 팀 워크스페이스로 복제하려면 → rootr.io