Testing denotes activities that aim at increasing confidence that actual and intended behaviors of a system conform, or at proving the assumption of conformance wrong. The relevance of testing is reflected in the general experience that it makes for about one half of the overall development cost. In this class, we convey advanced concepts and techniques and discuss assumptions as well as published evidence. In particular, we study the power of random and statistical testing, the promises of model-based testing, the intricacies of testing object-oriented software and advanced concepts for fault localization.

Modules taught:

1. Introduction
   1. Relevance
   2. Terminology and fundamental concepts
   3. Testing in the software development process
   4. Kinds of testing
   5. Cost vs. quality
   6. Overview of the class
2. Testing large parameter spaces
   1. Test selection with partitions and coverage criteria for control and data flows
   2. N-wise combinatorial testing
3. Random and statistical testing
   1. Random testing
      Analytical comparisons of random and partition-based testing: the models of Weyuker&Jeng and Gutjahr
   2. Statistical testing
      Operational profiles and Markov models
4. Model-based testing
   1. Models and abstraction: SUT/environment, state machines, block diagrams, contracts
   2. Scenarios and embedding in development process
   3. Generation technology
   4. Cost effectiveness, assumptions, and empirical evidence
5. State-based testing and testing OO software
   1. State-based testing for objects
   2. Testing with inheritance
   3. Testing with polymorphism
6. From failures to tests and faults
   1. State-based test extraction and capture&replay testing
   2. Delta debugging
   3. Recap: Inspections and walkthroughs
   4. Fault localization
7. Regression testing
8. Test assessment
   1. Fault injection and mutation testing for code and models
   2. Test selection for regression
9. Test specification (time permitting)
   1. TTCN-3
   2. Action-word based testing

Unfortunately, there is not one single book that covers all the material. I'll post some books later on. More references and links to research papers will be given in class.

Slides (available only from within KIT):

• Introduction (11.4.2011)
• Cause-effect graphs and coverage criteria (18.4.-21.4.2011)
• Testing Large Parameter Spaces (2.5.2011)
• Random and Statistical Testing (9.5.-30.5.2011)
• Model-Based Testing (30.5.-6.6.2011)
• State-Based Testing and Testing OO Software (20.6.2011)
• From Failures to Tests and Faults (27.6.-4.7.2011)
• Fault Injection and Mutation Testing (4.7.2011)
• A Note on Regression Testing (4.7.2011)
• Test specification (11.7.2011)
• Wrap-up (11.7.2011)

Exercises:

• Exercise1: Introduction, coverage criteria and cause-effect graphs (Issued 21.4.2011, to be submitted by 28.04.2011)
• Exercise2: Data flow analysis (Issued 29.4.2011, to be submitted by 5.5.2011)
• Exercise3: Combinatorial testing (Issued 6.5.2011, to be submitted by 12.5.2011)
• Exercise4: Random testing (Issued 14.5.2011, to be submitted by 26.5.2011)
• Exercise5: Random testing (Part II) (Issued 19.5.2011, to be submitted by 26.5.2011)
• Exercise6: Statistical testing and operational testing (Issued 26.5.2011, to be submitted by 2.6.2011)
• Exercise7: Operational profiles and Abstraction (Issued 2.6.2011, to be submitted by 9.6.2011)
• Exercise8: Model Based Testing (Issued 9.6.2011, to be submitted by 16.6.2011)
• Exercise9: Software Model Checking (Issued 16.6.2011, to be submitted by 23.6.2011)
• Exercise10: State-Based Testing and Testing OO Software (Issued 23.6.2011, to be submitted by 30.6.2011)
• Exercise11: Test extraction and delta debugging (Issued 30.6.2011, to be submitted by 7.7.2011)
• Exercise12: Fault localization, code review, and regression testing (Issued 7.7.2011, to be submitted by 14.7.2011)