Structural source code coverage and Requirements – Is there any dependency?

If you are newly engaged in the area of functional safety, then you will encounter fairly quickly the terms “structural source code coverage” and “requirements”. The specification of technical systems by requirements is, of course, also common in non-safety-critical areas. By contrast, the subject structural source code coverage is almost unknown outside safety-critical projects.
In this blog I want to discuss these two methods and the underlying processes. In particular, the question of inter-dependence is exciting.

Structural source code coverage:

The structural source code coverage is a measure of the degree of coverage of the source code by tests. There are different levels of coverage defined. Usually you start with the coverage of the executable source code lines. This means that if every line of code has been executed once, then you have reached 100% coverage.
With the famous MC/DC Coverage it is demonstrated that within a condition each parameter has an effect on the result of the decision. The minimum number of necessary test cases is the number of variables in a decision +. 1
The figure below shows different types of coverages:

Requirements Engineering:

Good functional requirements describe the behavior of a system under defined conditions. Functional requirements therefore provide ultimately the reasons for the implemented source code.
“Wait a moment!”
In the first sentence the term “system” is used. In the second sentence “the implemented source code” is used. Is there not a contradiction?
“Yes!”
This is a contradiction! With requirements, describing the system behavior, it is in many cases impossible to understand directly, a specific functional implementation in the software. This is so far still not a problem, since, software requirements should be derived from the system requirements. All standards of functional safety demand at least these two levels of requirements.
The requirements for the software should then provide the traceability to the implemented source code? Unfortunately, there are a lot of practical cases, where this traceability is not there. The reason for this is the ever-increasing complexity of software. The software architectures are becoming more complex in order to control the increasing functional complexity.
This has the unpleasant effect that in a lot of cases it is not even possible to trace from the software requirements to each function of the source code.
Since the structural source code coverage, should be measured based on the requirements based tests, a lot of projects tend to define at least one more level of requirements.
Now we have found the point at which the structural source code coverage influences the requirements engineering and vice versa.
According to my observation, the need to achieving 100% structural coverage increases the number of requirement levels in a lot of projects.

Conclusion:

The aerospace industry accumulated already 30 years of experience with 3 levels of requirements (system, high-level and low-level SW SW). These levels are even explicitly specified in the standard. The experience here is that there are very few projects, which have found a good balance between the different requirements levels. The aerospace industry demands also the achievement of 100% structural source code coverage.
Nevertheless, the creation of the third level of requirements, the low-level software requirements, is very complex and expensive. There are many projects where the benefit is quite doubtful.
Therefore, before you want to solve the problem, by creating 3 or more level of requirements, take a closer look on the problem. In most of the cases a thoughtful strategy, both for drawing up the requirements as well as for the test strategy to achieve the source code coverage, is clearly more productive and cost-effective than simply increasing the number of requirements.

Are you ready for a status workshop, to analyse improvement potentials in your test engineering, then send a mail to: info[at]heicon-ulm.de or call +49 (0) 7353 981 781.