This may sound strange, but I will prove it: no matter how big or stable a piece of software is, it has an unlimited number of bugs not yet found. No matter how many of them we have already managed to find and fix, there are still too many left to count.
Let’s take this simple Java method that calculates a sum of two integers as an example:
int sum(int a, int b) {
return a + b;
}
This simple program has an unlimited number of bugs.
To prove this claim we just need to put two thoughts together:
First, a bug is something that compromises the quality of software, which, according to IEEE 610.12-1990, is “the degree to which a system meets specified requirements or user expectations.”
Second, requirements and expectations may be functional and non-functional. The latter include performance, resilience, robustness, maintainability, and a few dozen other NFRs.
It is obvious that there are at least two variables in this equation that are ambiguous: user expectations and maintainability. We can’t be precise about them and that’s why the number of bugs they will produce has no limit.
Of course, only a very limited subset of the entire set of bugs has any real business impact. Most of the bugs that exist in a program may stay there even after it is shipped to its users—nobody will ever find them or else the damage they cause to the user experience will be insignificant.
Finally, take a look at the method sum() one more time. How about these bugs:
- It doesn’t handle overflows
- It doesn’t have any user documentation
- Its design is not object-oriented
- It doesn’t sum three or more numbers
- It doesn’t sum
doublenumbers - It doesn’t cast
longtointautomatically - It doesn’t skip execution if one argument is zero
- It doesn’t cache results of previous calculations
- There is no logging
- Checkstyle would complain since arguments are not
final
I’m sure you can find many more.
BTW, Glenford J. Myers said something very similar in his book “The Art of Software Testing,” which I reviewed earlier.
Bill Hetzel, The Complete Guide to Software Testing (1993): “Some Theoretical Limits to Testing: ‘We can never be sure the specifications are correct,’ ‘No testing system can identify every correct program,’ ‘We can never be certain that a testing system is correct.’ These theoretical limits tell us that there will never be a way to be sure we have a perfect understanding of what a program is supposed to do (the expected or required results) and that any testing system we might construct will always have some possibility of failing. In short, we cannot achieve 100 percent confidence no matter how much time and energy we put into it!”
