Traceability and audits

Nevertheless, it is found in the audit that many software requirements, are somehow related to the linked system requirements, but an integrated traceability strategy is not recognizable. If it is then determined in further audited examples that some aspects of system requirements are not implemented in the software (or hardware) the audit starts to be stressful. After this situation, you may also start thinking about the time and money you spent to achieve a good Requirement and Test Traceability. You almost inevitably raise the question now, what is the justification for the use of tools and engineering time, if the result is such poor. You may start thinking about a drastically reduction of time and effort in this area in the next project.

“Everything” to “Everything” traceability

However, instead of doing this, you should start a root cause analysis, to identify the real issues. Basically, the use of tools is very useful and there is also a significant added value if you do traceability correctly.
A major cause of the problems described is often the linkage of “everything” to “everything”, as a substitute for the use of a project-specific traceability strategy.
In former times, when there was no convenient tool support, in the creation of traceability, this problem just did not exist. The effort to create such traceability was far outside any acceptable range. Because of this fact, the projects were forced to make very intense thoughts about the following challenges:

1. Which results of the Hazard and Risk Analysis are implemented in which requirement(s)?
2. What are the software requirements which implement (really!) the considered system functionality (= System Requirements)?
3. Which Software Requirement is not derived from any system requirement, but arises as a consequence of architectural decisions and should therefore remain (=no link to another functional Requirement) as “Derived” Requirement?
4. Which source code can not be derived from functional requirements, but (e.g.) from the interface to the hardware?
5. Which source code occurs, for example due to applied principles as Defensive programming and not on the basis of functional requirements?
6. What tests actually verifies which requirement, really?
7. In which architectural block which requirements are implemented?

This list of questions does not claim to be complete.

But it forms a good starting point if you want to achieve a useful traceability.

Thanks to the support of modern and innovative Tools it is today much easier to put the individual artifacts in the software development process in an useful relation (Traceability) to each other. There are a variety of database-driven tools that reduce the complexity and the error rate for the traceability creation significantly. If you additionally make yourself aware of the issues described and you develop appropriate solutions, you are on the way to achieve the desired added value.

Traceability benefits:

1. Very effective mean to assure that the customer’s requested product was built accordingly.
2. Easy and quick navigation from the System Requirements to the Source Code/Hardware Implementation
3. In case of changes, effective support of required analysis to avoid unintended side-effects
4. Support / simplification of the completeness check with respect to the product implementation
5. Support / simplification of a completeness check with respect to the product verification
6. Good mean to track the implementation and verification progress

Releated HEICON Blog posts:

• Requirement Engineering – Aspects which even not considered in RE theory!
• Structural source code coverage and Requirements – Is there any dependency?
• DO-178B/C, ISO 26262, IEC 61508: How many level of Software requirements are necessary and useful?

Are you ready for a status workshop, to analyse improvement potentials in your requirement engineering, then send a mail to: info[at]heicon-ulm.de or call +49 (0) 7353 981 781.

Der Beitrag Requirement and Test Traceability – Any added value? erschien zuerst auf Heicon Ulm.

Further blog posts on related topics are:

• Tool qualification – The phantom pain of functional safety (part 2)!
• Tool qualification – The phantom pain of functional safety (part 1)!
• IEC 61508 – Tool qualification – When? Why? How?
• Compiler for safety critical software – What needs to be done?

Classification of software tools:

EN 50128 and EN 50 657 define the following three tool classes in Chapter 3.1:

Tool class T1: generate no outputs which can directly or indirectly contribute tot he executable code (including data) oft he software.

Examples: A text editor or a requirement or design support tool with no automatic code generation capabilities, configuration control tools.

Tool class T2: supports the test or verification of the design or executable code, where errors in the tool can fail to reveal defects but cannot directly create errors in the executable software.

Examples: test harness generator, a test coverage measurement tool a static analysis tool

Tool class T3: generates outputs which can directly contribute to the executable code (including data) oft he safety related system.

Examples: A source code compiler, a data7algorithms compiler, a tool to change set points during system operation, an optimising compiler where the relationship between the source code program and the generated object code is not obvious, a compiler that incorporates an executable run-time package into the executable code.

According to both standards, no further measures need to be taken for T1 tools.
T2 tools must essentially have a specification or manual in which the behaviour of the tool is clearly defined and all instructions or boundary conditions relating to the application are listed. In addition, the tool choise shall be justified and a configuration management process must be in place.
T3 tools shall also meet the following requirements:

• A suitable combination of history of successful tool use in similar environment
• Diverse redundant code which allows the detection and control of failures
• Compliance with the safety integrity levels derived from the risk analysis of the process and procedures including the tools
• Tool validation results
  • A record of validation activities
  • The version of the tool manual being used
  • The tool functions being validated
  • Tools and equipment used
  • The results of the validation activity
  • Test cases and their results for subsequent analysis
  • Discrepancies between expected and actual results

Practice assessment:

For experienced functional safety experts, EN 50128 and EN 50657 define a very good framework. However, project practice shows that there are still many questions.

EN 50128 and EN 50657 support tools request, more strongly than other functional safety standards, the black box test of the tool used. These tests must be documented (test cases, test procedures, test results). The required specification of the tool is not clearly defined. A requirement specification and the user manual are evaluated more or less equally. The effort for a specification with atomic, testable requirements is usually time-consuming and expensive. The user manual is supplied free of charge with the tool. No direct statement is made about the quality or level of detail of the traceability between specification and tests.

A further aspect in practice is that the manufacturers of commercial tools often offer so-called tool qualification packages. These packages are usually attractive, because the tests and the corresponding documents are already finished and you “only” have to execute the tests once in your environment. However, in this case you have hardly any influence on the content of the delivered tool qualification package. From the point of view of the assessor, however, you are still responsible for the content, i.e. you must know and evaluate the content.

Practical guide for the use of supporting tools according to EN 50128 and EN 60567:

Regardless of who creates the qualification package tool, the following is recommended:

1. creating an overview of all software tools used in the project with a justification of the tool class (T1, T2, T3). For the vast majority of tools, the result will be T1.
2. creation of functional requirements describing the behavior of the tool
3. creation of test cases which describe what the following test specifically tests
4. traceable and plausible linking between requirements and test cases
5. documentation of requirement and test cases in a database
6. creation of automated test scripts including definition
7. use of a professional configuration and change management system

With this procedure you will be able to successfully use 80% – 90% of all software tools relevant in practice. I have described the procedure for the compiler in the blog Compiler for security-relevant software. I would like to make a further restriction for tools which can directly generate errors in an executable SIL3/SIL4 software. Even if the standard here is not so clear, it can be good in practice that further measures are required here.

I’ll be glad to help you also with any specific questions about your project. Send an email to: info [at] heicon-ulm.de

An overview of the HEICON services can also be found on the HEICON Homepage.

Der Beitrag EN 50128 and EN 50657 support tools erschien zuerst auf Heicon Ulm.