Chapter 14

Chapter 14

Software Testing Techniques

CHAPTER OVERVIEW AND COMMENTS

This intent of this chapter is to introduce a variety of black-box and white-box testing methods that can be used for both conventional and object-oriented software. The vast majority of students will be unaware of even the most simple test case design method, viewing testing as an afterthought—something that has to be done after coding. Students should be encouraged to design and build test cases for their course projects, using several of the testing techniques presented here.

14.1  Software Testing Fundamentals       

Students need to be encouraged to look at testing as an essential part of the quality assurance work and a normal part of modern software engineering. Formal reviews by themselves cannot locate all software errors. Testing occurs late in the software development process and is the last chance to catch bugs prior to customer release. This section contains a software testability checklist that students should keep in mind while writing software and designing test cases to test software. The toughest part of testing for students is understanding the necessity of being thorough, and yet recognizing that testing can never prove that a program is bug free.

14.2  Black-Box and White-Box Testing

This section discusses the differences between black-box and white-box testing. Another purpose of this section is to convince students that exhaustive testing is not possible for most real applications (too many logic paths and two many input data combinations). This means that the number of test cases processed is less important than the quality of the test cases used in software testing. Sections that follow discuss strategies that will help students to design test cases that will make both white-box and black-box testing feasible for large software systems.

14.3  White-Box Testing

This section makes the case that white-box testing is important, since there are many program defects (e.g. logic errors) that black-box testing can not uncover. Students should be reminded that the goal of white-box testing is to exercise all program logic paths, check all loop execution constraints, and internal data structure boundaries.

14.4  Basis Path Testing

This section describes basis path testing as an example of a white-box testing technique. Basis path testing is easiest for students to use if they will construct a program flow graph first. However, students should understand that cyclomatic complexity could be computed from the PDL representation of the program (or from source code itself). Be sure your students understand what an “independent path” is and why there are a limited number of them (as opposed to an extremely large number of program paths).

Students should be encouraged to use the basis path example as a model and construct a set of test cases for one of their own programs. The term "graph matrix" is introduced in this section; students might have studied these as adjacency matrices in a discrete mathematics or data structures unit on graph theory. If your students are unfamiliar with graph theory, you may need to show them more examples of how to construct adjacency matrices with various types of graph edge weights.

14.5  Control Structure Testing

Basis path testing is one form of control structure testing. This section introduces three others (condition testing, data flow testing, loop testing). The argument given for using these techniques is that they broaden the test coverage from that which is possible using basis path testing alone. Showing students how to build truth tables may be beneficial to ensure thorough coverage by the test cases used in condition testing. Students may need to see an example of building test cases for data flow testing using a complete algorithm implemented in a familiar programming language. Similarly students may benefit from seeing examples of building test cases for each of the loop types listed in Section 14.5.3.  Students should be required to build a set of test cases to do control structure testing of one of their own programs sometime during the semester.

14.6  Black-Box Testing

The purpose of black-box testing is to devise a set of data inputs that fully exercise all functional requirements for a program. Students should be reminded that black-box testing is complementary to white-box testing. Both are necessary to test a program thoroughly. Several black-box testing techniques are introduced in this section (graph-based testing, equivalence partitioning, boundary value analysis, comparison testing, orthogonal array testing).

It is important to emphasize that in black-box testing the test designer has no knowledge of algorithm implementation. The test cases are designed from the requirement statements directly, supplemented by the test designer's knowledge of defects that are likely to be present in modules of the type being tested. It may be desirable to show students the process of building test cases from an actual program's requirements using several of these techniques. A worthwhile activity for students is devising test cases for another student's program from the software specification document without seeing the program source code.

14.7  Object-Oriented Testing Methods

Test case design for OO software is directed more toward identifying collaboration and communication errors between objects, than toward finding processing errors involving input or data like conventional software testing. Fault-based testing and scenario-based testing are complementary testing techniques that seem particularly well-suited to OO test case design. White-box test case construction techniques are not well suited to OOT. Students should be encouraged to develop a set of test cases for an OO system of their own design. Students need to be reminded of the fact that the process of inheritance does not excuse them from having to test operators obtained from superclasses (their context has changed). Similarly, operators redefined in subclasses will need to be tested in scenarios involving run-time resolution of the operator calls (polymorphism). Students should spend some time discussing the differences between testing the surface structure (end-user view) and deep structure (implementation view) of an OO system.

14.8  Testing Methods Applicable at the Class Level

This section discusses the process of testing at the individual class level. Students should be reminded of the haphazard nature of random testing and be urged to consider using the three operation partitioning techniques (state-based, attribute-based, category-based) to improve the efficiency of their testing efforts.

14.9  Interclass Test Case Design

This section discusses the task of interclass test case design. Two techniques for conducting interclass testing are described (multiple class testing and tests derived from behavioral models). Students might be encouraged to develop a set of test cases for a real system using each technique and see which they prefer using.

14.10   Testing for Specialized Environments, Architectures, and Applications

This section briefly discusses several specialized testing situations (GUI's, client/server architectures, documentation and help facilities, real-time systems). More extensive discussion of these testing situations appears elsewhere in the text or in the SEPA web site resource links. Testing of Web applications is considered in Chapter 20.

14.11   Testing Patterns

Like their counterparts in analysis and design, testing patterns describe situations that software testers may recognize as they approach the testing of some new or revised system. If time permits, ask your students to research testing patterns on the Web and present a pattern to the class.