Using the Scientific Method and Bisection for Debugging Software
Scientific Method
- Study available data (test results, and program text with a critical eye). Ask: what could have gone wrong?
- Form a hypothesis as to why the program did not behave it should have based on the analysis of available data
- Design and run a repeatable experiment (write a test)
- The experiment must have the potential to refute the hypothesis - If the experiment is not capable of proving the hypothesis incorrect, it is of no use!
- The expected (correct) result of the experiment must be established before the test
- Run the experiment and check if it meets the expectation
Debugging
- Approach debugging systematically
- The objective is to reduce the search space - i.e. amount of the source code and data we need to look at to find the source of problem
- Use bisection (as used in binary search) to reduce the search space. Start around the middle of search space (source code and/or data).
- Establish in which section of the bisected search space does the root cause of the problem lie
- Repeat the bisection process in the selected section
- By using this approach we reduce the search space into half with each attempt thus getting to root cause in a systematic, and efficient manner
What to look for
- Order of arguments passed to a method
- Initialization of variables, especially the ones used in a loop. Did you intend to initialize these inside or outside the loop?
- Comparison of objects and values. Did you intend the reference (address) or value?
- Are multiple variables pointing to the same object and perhaps changing the object.
- Look for side-effects. Is the method being called changing the arguments passed in?
- When copying objects, did you intend to make shallow or deep copy of the object?
- Do not exclusively rely on the code comments. Read the code.
- Reconsider the assumptions. Are you looking at the correct part or version of the code? Are you using the right data to reproduce the problem?
- Walk away and take a fresh look later
- Don't rush to fix the bug. Look closely at the nature of the bug. Is is spread among other portions of the code. Does fixing this bug introduce other bugs? Are you adding a lot of code to fix the bug? Should you first clean up and refactor?
