In the first part of the blog I defined the term Implicit Testing and discussed root causes for the need of implicit tests. Now, in the second part I will focus on the disadvantages of such tests and on possible solution approaches with the goal to avoid these disadvantages.

Disadvantages of Implicit Testing or Indirect testing:

When analyzing projects at a later stage, then it will be realized that there is a link between implicit tests and the number of errors that have not been found. Through implicit testing, a essential strength of requirement-based testing is lost. This is that small functional portions are defined in requirements and completely tested. Implicit testing does not do this anymore. Often, entire requirements are no longer explicitly tested. This can also be seen in the example requirement from the first part of the blog:

Req. A: If Weight on Wheels = TRUE is present at Discrete Input DS 1, the software flags shall be set as follows: SWW1 = TRUE and SWW2 = FALSE.

Req B: If SWW1 = TRUE and SWW2 = FALSE then the AFDX Msg “ON_GROUND shall be sent.

Because of the use of the wrong or inadequate test environment it is not possible to prove explicitly whether the internal software variable really are set as defined in the requirement A. Similarly, it will not be explicitly proven that the variables must be set accordingly in order to determine the reaction according to Req. B.
With increasing system complexity, there is a risk that errors in the system will no longer be found by the test.
In addition to an inadequate test environment, the cause of implicit testing can also be the attempt to prove the architecture of a system by tests. Instead of defining the architecture graphically, textual requirements are often created, which are only implicitly testable. From cost-benefit aspects, it is a fact that such types of textual requirements are mostly useless. Without this kind of requirements, the same product quality would have been achieved, but at a lower cost.
Technically speaking, the advantage is that if such requirements are not available, the documentation becomes clearer, simpler and more transparent.
In general, many projects today tend to define too many requirements rather than too few requirements. Architectural descriptions in the form of requirements, as well as the definition of requirements at the wrong level are the main reasons for that.

Possible solution approaches:

The following list defines a rough overview of solutions to minimize implicit testing and thereby significantly increase the efficiency and the cost / benefit factor in the project:

• Conscious strategy for the creation of requirements already at the beginning of the project
  • Dealing with different categories of requirements
  • Definition of requirements at the appropriate level
  • For 90% of all embedded systems valid: define a maximum of 2 levels of requirements
  • Individual determination of the test level or test environment for each requirement
• Definition of an efficient test strategy already at the beginning of the project
  • Definition and conscious use of a system test environment
  • Definition and conscious use of an HW / SW integration test environment
  • For projects with a high amount of software or for safety-relevant projects Definition and conscious use of a SW / SW integration test environment

Related HEICON Blog posts

• Implicit Testing – A good idea (Part 1)?
• Testing of Platforms: A challenge!
• Management aspects of testing
• Psychology of Testers

Are you ready for a status workshop, to analyse improvement potentials in your test engineering, then send a mail to: info[at]heicon-ulm.de or call +49 (0) 7353 981 781.