Unit 8 Study Guide and Review: Software Testing
8a. Describe basic software testing terminologies
- What is software testing?
- When is software testing performed?
- Where is software testing performed?
- Why is software testing performed?
- How is software testing performed?
- Who performs software testing?
What: Software testing is a process of executing software to find errors. It is a type of verification. Other types of verification include reviews and analysis. If the software satisfies its specification, it is said to be verified. If it satisfies the user requirements, it is validated. Errors are either errors of omission or errors of commission. Projects also usually categorize errors by severity and by product, such as design errors, documentation errors, level 1 and 2 errors, and so on.
When: Software testing is performed primarily in the testing phases of development, after the implementation phase. However, testing is also done in other phases, such as the design phase, with the aid of scaffolding software. Scaffolding software is test software (not part of the application software) that is used to provide an environment that enables partial code to be executed. Testing stops when all test cases and procedures have been successfully completed.
Where: Software testing can be performed in a development environment, a test environment, or in an operation environment.
Why: Software testing is performed to find errors in the software and to verify and validate the software.
How: Software testing is performed using test cases, test procedures, test data, operational data, test software, and test tools. Further, the testing is conducted per a test plan and test process.
Who: Testing is performed by the developers, by project testers, by internal company testers, by external testers, and by customer testers.
To review, read pages 690-694 of The New Software Engineering.
8b. Compare and contrast the use of various testing strategies, including black-box, white-box, top-down, and bottom-up
- Define software testing strategies.
Four major testing strategies are black box, white box, top down, and bottom up. A module has three parts: input, process, and output. A strategy for testing software includes a way of selecting the test data.
If the test data selected is input data based on design specifications and not on the logic of the software, the test strategy is called black box testing. In black box testing, a module is executed with the test or input data and the output is compared to the required or expected output. If they are the same, the test was successful; if not, the test failed. This is called black box testing because the processing part of the module is not utilized in selecting the test data, and is instead treated as hidden.
If information on the module's processing is used to select the input data, the test strategy is called white box testing. For example, suppose a module has two execution paths controlled by a binary condition. In white box testing, test data would be selected to exercise each path. In general, a module could have a huge number of execution paths, and analysis would determine the test data to target which paths to exercise.
The design of software consists of a collection of modules organized as two structures, static and dynamic, in the form of a module hierarchy. For the static structure, the relationships reflect the parts of a module. For the dynamic structure, the relationships reflect the execution paths of the collection. In bottom up testing, testing begins when the coding of a module at the bottom of the hierarchy has been completed. When all of the parts of a module have been coded and tested, the parent module is then tested. Thus, the order of testing proceeds from the bottom of the hierarchy to the top.
In top down testing, the order of testing proceeds from the top of the hierarchy down, along execution paths. In order to execute a module when only a part of that module has been coded, some test code called scaffolding may have to be written. This test code is in addition to the module code and will be replaced by the module code when it becomes available. To review, read pages 691-694 of The New Software Engineering.
8c. Design a test plan to include unit, integration, and system levels of test coverage
- What is included in a software test plan?
- What are the activities of a software test phase?
- In addition to verification and validation, are there other benefits from the test activities?
- There are several types of tools available to support software testing.
- Compare testing in data-oriented, process-oriented, and object-oriented methodologies.
- How are tests verified and validated?
Testing is a critical, costly, and challenging activity that verifies and validates the code, software design, and software requirements. The requirements, design, and implementation phases have corresponding levels of test activities: system, integration, and unit, respectively. Each level of testing is conducted according to a test plan. A test plan addresses why testing is done (test goals); who does the testing; what is tested; when testing starts, stops, and is redone (regression); where testing occurs; and how it is done. System testing and quality assurance testing are similar and share test strategies, cases, and scripts. However, quality assurance testing is a pure validation test. A variety of software tools are available to support the test activities, and those selected are identified in the test plan.
Testing produces benefits in addition to improving the quality of the software. Testing produces information and measures about the application system that will be helpful in making management and technical decisions during further development and maintenance. For example, traceability information, identifies the impact of errors on other parts of the system and helps determine the scope of related changes, the effort needed for regression testing, and complexity of the system. Moreover, defect measures from tests, along with technical reviews, can be used to determine the probability of remaining errors and to target test resources to the parts of the software that have the greatest risk.
Do data-oriented, process-oriented, and object-oriented methodologies differ or are they the same with respect to testing? Logically the activities are the same. The artifacts and diagrams used by the respective activities are different, particularly, in the emphasis on data, process, or objects.
Testing is a recursive activity, in that tests need to be verified, validated, and themselves tested. Test code used for the first time should be tested, particularly for critical software, in the same way that the system code is tested. Historical performance of the test software, reviews of test case coverage, and analysis of test results or errors, either found or not found, are also used to verify and validate the test software.
Here is a diagram that summarizes the Testing Phases:
Unit 8 Vocabulary
This vocabulary list includes terms that might help you with the review items above and some terms you should be familiar with to be successful in completing the final exam for the course.
Try to think of the reason why each term is included.
- Test data
- Verification
- Validation
- Software test
- Error types
- Top down testing
- Bottom up testing
- Black box testing
- White box testing
- Regression testing
- Regression test package
- Test case
- Test script
- Test procedure
- Execution path
- Input-process-output
- Conformance
- Test scaffolding
- Test driver
- Test benefits