ISO 26262 Confidence in the use of software tools – A feasible strategy!

ISO 26262 Confidence in the use of softwar tools: ISO 26262-8 in chapter 11 defines the requirements for software tools that are used in an ISO 26262 compliant software development process. In the practical usage of chapter 11 of ISO 26262-8 many questions often arise, which partly lead to very creative approaches regarding tool qualification.

The following article clarify open questions in this subject and provides answers to the following 3 questions:

Why does ISO 26262 require a proof of confidence in the tools used at all?
What are the requirements of ISO 26262?
How to deal with the ISO 26262 Confidence in the use of softwar tools topic?

Why does ISO 26262 require a proof of confidence in the tools used at all?

All functional safety standards can be generalized to the use of the 4-eye principle in the area of development processes. A second instance/person checks each activity defined in the safety standard. The use of software tools can violate this principle. This can be clearly illustrated using static code analysis tools. Such a tool is used to check the compliance of the defined coding guidelines in the source code. If the tool contains an error, any violations of the corresponding coding guidelines are not detected because no further verification takes place.

The ISO 26262 Confidence in the use of softwar tools fits exactly into this gap. Correctly applied, it restores the 4-eye principle for the area of software tools.

ISO 26262 formulates this as follows:

A software tool used in the development of a system or its software or hardware elements can support the activities and tasks required by ISO 26262. In such a case, confidence is needed that the software tool will effectively achieve the following objectives:

the risk of systematic errors in the developed product due to malfunctions of the software tool leading to incorrect outputs is minimised
the development process is adequate with respect to compliance with the ISO 26262 series of standards if activities or tasks required by the ISO 26262 series of standards rely on the correct functioning of the software tool used

What are the requirements of ISO 26262?

First of all, according to ISO 26262, the Tool Confidence Level must be defined for a specific tool. For this purpose the standard defines the following table:

The Tool Impact Level and the Tool Error Detection Level are defined as input parameters. The standard defines these parameters as follows:

TI1: Tool malfunction can not introduce or fail to detect errors in a safety-related item or element being developed.
TI2: If TI1 does not apply

TD1: High degree of confidence that a malfunction and its corresponding erroneous output will be prevented or detected
TD2: Medium degree of confidence that a malfunction and its corresponding erroneous output will be prevented or detected
TD3: shall be selected in all other cases

If either TD1 or TI1 is obtained for a particular tool, this results in a TCL1. For tools that are classified in this way, no further action is required. This is mostly true for e.g. text editors, requirement databases and configuration management tools.

If for a tool a TD2 and a TI2 is obtained, this results in a TCL2. Similarly, a TD3 and TI2 results in a TCL3.

Methods recommended by ISO 26262

For TCL2 and TCL3 the ISO26262 defines the following 4 possible methods that can be used:

For ASIL A and B, the standard, whether TCL2 or TCL3, prefers the methods “Increased confidence from use” and “Evaluation of the tool development process”.

For tools with TCL2, this preference also applies to ASIL C projects.

For ASIL C and D, the standard recommends the methods “Validation of the software tool” and “Development in accordance with a safety standard” for tools that have a TCL 3.

For tools that have a TCL 2, and are used in a ASIL D project the norm prefers the method “Validation of the software tool”.

What do the individual methods mean?

Increased confidence from use

This method requires that a tool has already been used in comparable projects and has been on the market for many years. At the same time, the errors that have occurred must be accessible and suitable for the application.

Evaluation of the tool development process

Here the user of the software tool evaluates the development process of the tool, if necessary at the tool vendor side. In practice, it has been established that this method is almost only used if an independent body has issued a certificate for the evaluation of the development process and this is accessible.

Validation of the software tool

The tool user or vendor creates requirement based, functional tests for the used functions of the software tool and by executing these tests he proves the correct operation of the software tool. Ideally, these tests always take place in exactly the same environment in which the tool is used.

Development in accordance with a safety standard

Using this method, the entire software tool is developed according to a safety standard such as ISO 26262, DO 331 etc. The level of criticality used is the same as the one used for the software for which the software tool is to be developed. The developer of an ECU software, which he develops according to ISO 26262 ASIL D, must accordingly prove that the manufacturer of the software tool used in the development process has also developed it according to ISO 26262 ASIL D. This means a significantly increased development effort.

How to deal with the ISO 26262 Confidence in the use of softwar tools topic?

The major challenge in tool qualification according to ISO 26262 lies at the beginning of the process, the determination of the tool confidence level. The standard leaves a lot of room for interpretation here. In the first step I recommend the following procedure:

Does the tool under consideration replace or automate a process defined in ISO 26262 and can the tool insert an error into the operational software ( possible example: code generator)?
Does the considered tool replace or automate a process defined in ISO 26262 and can the tool not find errors that are in the operational software (example: static code analysis tools, unit test tool etc.)
If both questions under 1) are answered with “Yes”, only the measure “Development in accordance with a safety standard” is applicable. In practice, however, this is the absolute exception, as it is often forgotten that ISO 26262 processes also need to be replaced. This would be the case, for example, if unit tests and/or code reviews were to be omitted for source code created by a code generator. In general, the creation of these artifacts is much cheaper than the development of the code generator according to an ASIL level of ISO 26262. Also the compiler does not have to be qualified, because no process from ISO 26262 is replaced. Details can be found in the blog Compiler for safety-relevant software – What to do?
If both questions under 2) are answered with “Yes”, then for ASIL C and D the measure “Validation of the software tool” makes most sense. Many years of experience in the application of this measure show that it is the most efficient tool qualification method. The effort is also not excessive in most cases. For many commercial tools many manufacturers even offer a so-called tool qualification package. For ASIL A and B projects this measure is also recommended in principle, but from the point of view of the standard a certificate from the manufacturer on the evaluation of the tool development process is sufficient.
If either 1) or 2) can answer in the negative to the question of automation or replacement of an ISO 26262 process with a tool, then no further action is necessary. This is usually the case for the majority of tools used, such as text editors, requirement databases, configuration management tools, etc.