Investigating Bugs

---
name: investigating-bugs
description: Use when encountering any bug, test failure, build error, or unexpected behavior.
---

# Investigating bugs

Avoid "guess-and-check" coding. Always identify the root cause before making changes.

> **THE IRON LAW:** NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST.

> **Violating the letter of the rules is violating the spirit of the rules.**

> **REQUIRED PREREQUISITE:** You must have already completed `slow-powers:working-in-isolation` — debug from an isolated workspace.

> **REQUIRED BACKGROUND:** You must understand `slow-powers:working-with-tdd` — Phase 4 captures the bug with a failing test before fixing it.

---

## Phase 1: root cause investigation

Complete these steps before changing any code:

1. **Read error messages and stack traces:** Read every line of the error. Note the exact file, line number, and error codes.
2. **Reproduce consistently:** Identify the exact steps, inputs, or environment needed to trigger the bug. If it cannot be reproduced, gather more logs instead of guessing.
   * For flaky tests (pass sometimes, fail under load or only in CI), find the non-determinism before changing anything — don't rerun until it goes green. Arbitrary `sleep`/timeout delays are one common cause (wait on the actual condition, not a guessed duration — see [condition-based waiting](references/condition-based-waiting.md)); a dependency called a non-deterministic number of times or in a non-deterministic order is another. Read [diagnosing flaky tests](references/diagnosing-flaky-tests.md) for the general diagnostic method and cause catalog.
3. **Check recent changes:** Run `git diff`. Analyze recent commits, dependency additions, or configuration changes.
4. **Gather evidence in multi-component systems:**
   * Log inputs and outputs at every component boundary.
   * Instrument the layers step by step (for example, workflow → build script → runtime → database) to pinpoint exactly where the state breaks.
5. **Trace data flow:** Trace variables backward from the failure point to their source. Fix the bug at the source, not the symptom.
   * When manual tracing dead-ends, instrument the suspect operation: log the key inputs, relevant environment, and a captured stack trace (`new Error().stack`) *just before* it runs. In tests, write to stderr — a logger may be suppressed. Read the captured stack to find the original caller, then remove the instrumentation.

---

## Phase 2: pattern analysis

1. **Find working examples:** Search the codebase for similar logic that functions correctly.
2. **Compare implementations:** Identify every difference between the working version and the failing version. Do not assume "that difference doesn't matter."
3. **Verify dependencies and configuration:** Ensure all required modules, configuration, and environment variables are present and correct.

---

## Phase 3: hypothesis and testing

1. **Formulate a single hypothesis:** Write down a clear statement: *"I think X is the root cause because Y."*
2. **Test minimally:** Make the smallest possible change to verify the hypothesis (for example, add a log or change one value).
3. **Verify and reevaluate:** Did the test prove your hypothesis?
   * **Yes:** Proceed to Phase 4.
   * **No:** Revert the test change completely and formulate a *new* hypothesis. Never stack guess-on-guess.

---

## Phase 4: implementation and verification

1. **Write a failing test case:** Create an automated test or simple script that consistently triggers the bug. Verify it fails.
2. **Implement the fix:** Make a single, targeted change that directly addresses the root cause. Do not bundle unrelated refactoring.
3. **Verify the fix:** Run the test suite. Ensure the new test passes and no regressions are introduced.
4. **Apply the three-fix limit (architectural check):**
   * If you attempt **three separate fixes** and the bug remains: **STOP.**
   * This is a strong signal that the issue is architectural (e.g., wrong model assumptions, coupled state, race conditions).
   * Re-evaluate the system architecture and discuss the approach with the user before attempting a fourth patch.

---

## Common rationalizations

| Excuse | Reality |
|--------|---------|
| "This is an emergency, we don't have time" | 5 minutes of investigation beats 5 hours of chasing symptoms. |
| "I can see the symptom fix is obvious" | Obvious symptom fixes hide the real root cause. |
| "We tried three things, just add one more" | Shotgun fixes create new bugs. Stop and re-analyze. |
| "The senior engineer says this is the fix" | Authority is not evidence. Verify the hypothesis. |
| "We need to ship now, investigate later" | "Later" investigations never happen on shipped code. |
| "This case is different because..." | It is not different. The process applies. |

---

## Red flags — STOP and reset

- Writing a fix before reproducing the bug or reading the full stack trace
- "Let's just try changing X to see if it works"
- Stacking multiple speculative fixes on top of each other
- Claiming a bug is fixed without running the verification test suite
- Each "fix" only shifts the bug to a new location

All of these mean: STOP. Revert changes. Return to Phase 1.