Structural Source Code Coverage: Are you working in software projects where functional safety is becoming more and more important? The use of IEC 61508, ISO 26262 or a comparable standard is around the corner or you are already in the middle of such a project? In these cases you have probably already encountered the term Structural Source Code Coverage. Many people often consider the achievement of structural coverage to be equivalent to a very high testing effort and they doubt whether the costs and benefits of this topic are in a reasonable ratio.

What is structural source code coverage

In software development relevant to safety, a fundamental distinction is made between two types of coverage. The functional or requirements coverage describes how many requirements have been verified by tests. Due to the mostly natural language requirements and test case descriptions, no mathematically exact information can be given here.

This is different with structural source code coverage. Here it is measured how much of the source code is verified by tests. Software tools clearly determine the source code coverage because it is based on mathematical rules.

Ideally, source code coverage is always determined in requirements-based tests. Thus, the source code coverage can be used as a quality measure for requirements and tests.

Common errors in the determination of the source code coverage

In practice, many people often make the following two mistakes:

The determination of the total coverage is distributed incorrectly over the individual test levels.
The measurement of the source code coverage is not fully automated

Some experience is required to correctly distribute the determination of coverage among the test levels. Usually the coverage is determined exclusively at unit test level. This is a big disadvantage, because the connection of the tests to the functional software requirements is small.

On the other hand, if you try to achieve full source code coverage at integration or system level, the test effort increases almost exponentially for the last 20% of coverage.

For a complete automation of the source code coverage, there is often a lack of attention to detail when creating the test environments. If the test environment is not carefully planned and built at an early stage, the automation of the coverage measurement suffers. In many projects this leads to the fact that coverage is measured only once at the end of the project.

The following picture shows the connection between the test levels and the structural coverage that can be achieved in an efficient way:

Advantages if the source code coverage is determined in a focused way

A good approach is to measure the coverage achieved by the already existing functional tests on HW/SW or SW/SW integration level. Experience shows that the coverage of such tests is often 50% to 80%. Through good project management it is then possible to test only the remaining 20% to 50% at unit test level. The goal should be that the achievement of coverage does not lead to additional testing effort, or coverage measurement should be used to identify gaps in the requirements and functional tests.

If the coverage measurement is fully automated, especially on the integration test level, there are practically no additional costs. Thus, the use of coverage measurement can even be interesting for projects that do not have to meet a functional safety standard.

Related HEICON Blog posts

• Challenges when determining the structural source code coverage on the target!
• Structural source code coverage and Requirements – Is there any dependency?
• The non-intrusive measurement of structural coverage!

Are you ready for a functional safety workshop, to analyse improvement potentials in your development process, then send a mail to: info[at]heicon-ulm.de or call +49 (0) 7353 981 781.