HW14: Testing Reflections

Testing

The first part of this paper is about the psychological effects testing can have on the programmer where it is thought that a good programmer should have no bugs in their code.  And when we do discover bugs in our code then we are bad programmers with all the psychological ramifications that carries with it.  Apparently, some statistic show that per 100 lines of code 1-3 bugs is possible.  Then there’s an explanation on how software differs from making physical widgets (which was a little confusing because nowadays a widget can very much be understood as a piece of software).  With manufacturing the costs are in the manufacturing process whereas in software the costs are in the development.

There is the part where the author explains the several phases of attitudes towards testing ranging from thinking it’s merely to discover bugs (phase 0) to making it second nature as part of what we do in software development, to think of testing seriously so we don’t spend much effort on it and produce low-risk software.  I agree that most times we are somewhere in the middle.  Instead of thinking “design, code, desk check (?), test and debug” we should be thinking “design, test, design, code, test code, program inspection, test inspection, test debugging, test execution, program debugging, testing.”  C’mon, I struggled to write this out let alone actually follow this through in development unless it’s safety critical… I’m clearly in phase 2.5 of 5…

The pesticide paradox was interesting because it makes sense in the natural world that killing bugs in this manner leaves only those bugs resistant to it and we are forever looking for stronger and newer poisons but not really treating to root cause.  In software what that leaves us with are just subtle bugs that are very complex and difficult to solve.  There’s no silver bullet after all.

We get into a toy example of how a software project might be carried out and to various myths we usually have towards testing until we arrive at the part of how testing should really be carried out and it is somewhat similar to the book.  This is done in order by first unit testing usually as the developer, then component testing which looks at functional specifications, integration testing which looks at collections of components to make sure these are working correctly, and finally system testing which gives us some NFRs to worry about like performance, security, accountability, etc.  This last one is really just one big component.

Finally, I can agree with the author’s point and also a point made in our testing chapter that proving that a program is bug free is nearly impossible.  That automating debugging, also nearly impossible.  Which is why we must treat testing more like billiards than kiddie pool where you just hit the balls and those that fall are our “expected outcomes.”  You have to call your tests, inputs and outputs — no slop.