Root Cause Analysis (RCA) Template
Use this when a postmortem's root cause section needs more rigor than a paragraph. Think recurring incidents, cross-team failures, or anything where the first explanation ("the deploy broke it") is clearly not the whole story.
This is a companion to the postmortem template, not a replacement. The postmortem covers the incident narrative.
This covers causal analysis in depth.
How to use this document
- Start from the triggering event and keep asking "why" until you hit something you can actually act on (a missing safeguard, a wrong assumption, a process gap), not a person or "human error."
- Involve the people who were actually there. Causal chains reconstructed by someone who wasn't in the incident tend to be wrong in subtle ways.
- If two people disagree about the root cause, that disagreement is itself useful data, write down both views and dig into why they differ.
Problem statement
[One or two sentences: what happened, observed effect, when. No causes yet, just the symptom.]
Example: Checkout success rate dropped from 99.6% to 91% for 23 minutes on 2026-03-04.
Five Whys (or as many as it takes)
Push past the first plausible answer. Each "why" should be a factual, verifiable statement.
If you can't verify it with a log, metric, or code diff, that's a sign you're guessing.
- Why did checkout success rate drop?
→ [Answer 1]
- Why did that happen?
→ [Answer 2]
- Why did that happen?
→ [Answer 3]
- Why did that happen?
→ [Answer 4]
- Why did that happen?
→ [Answer 5, this is usually where you land on something actionable: a missing test, a design assumption, a monitoring gap]
Stop when you reach something your team can actually change. "Why did the engineer not notice" is rarely a useful place to land.
Keep going until you hit a system or process cause.
Contributing factors
Root cause is rarely singular. List everything that had to be true for the incident to happen.
If any one of these were different, would the incident have been avoided or smaller?
| Factor | Type | Detail |
|---|---|---|
| [e.g. No backoff cap in retry logic] | Code | [detail] |
| [e.g. No alert on connection pool saturation] | Monitoring | [detail] |
| [e.g. Deploy went out on a Friday afternoon] | Process | [detail] |
| [e.g. Staging load is 1/100th of prod, so this never reproduced] | Environment | [detail] |
Causal chain diagram (optional but recommended)
A simple linear chain is often enough. Use a Rootr mermaid block or whiteboard node for anything with branches.
Downstream payment provider latency spike (external, ~800ms p99)
→ retry logic re-attempts without backoff cap
→ connection pool exhausted within 90 seconds
→ new checkout requests time out waiting for a connection
→ checkout success rate drops to 91%Why didn't we catch this earlier?
This is the section that's most often skipped, and it's the most valuable one. Was there a test that should have caught it?
A code review comment that got dismissed? A similar near-miss nobody looked into?
[Answer honestly. "We had a similar minor blip three weeks ago and didn't investigate it" is exactly the kind of finding this section exists for.]
Verification
How do you know this is actually the root cause and not just a convenient story? What would disprove it?
- Reproduced the failure mode in staging/test environment
- Confirmed via logs/metrics that the proposed mechanism matches the observed timeline exactly
- Ruled out alternative explanations: [list what else was considered and why it was ruled out]
Corrective actions
Split these into "fixes the root cause" and "reduces blast radius or detects it faster next time." Both matter, but don't let the second category substitute for the first.
| Action | Addresses | Owner | Due |
|---|---|---|---|
| Add capped exponential backoff to payment client retries | Root cause | [name] | [date] |
| Alert on connection pool utilization >80% | Faster detection | [name] | [date] |
| Load-test staging at realistic traffic multiples quarterly | Prevents blind spot | [name] | [date] |
Example: filled Five Whys
Why did checkout success rate drop? Checkout API pods ran out of database connections.
Why? The payment provider client retried failed calls in a tight loop, and each retry held a connection open longer.
Why did retries hold connections that long? No backoff or timeout cap was set on the retry wrapper introduced in last week's client library upgrade.
Why wasn't that caught in review? The upgrade PR changed a config default. The diff didn't make the behavior change obvious, and there's no test that exercises retry behavior under downstream latency.
Why is there no test for that? The team's test suite covers success and failure paths but not degraded-latency paths. This class of failure has never been treated as a first-class test scenario.
이 템플릿을 팀 워크스페이스로 복제하려면 → rootr.io