AgQua 2016-2018: Aggregation Procedures for Quality

Motivation

In the automotive sector, it is essential that the authorization and release of software is based on a sound assessment of its quality. Not only can low software quality lead to high costs and damage to a brand's image as a result of product recalls, faulty software can, in worst-case scenarios, cause life-threatening situations.

To this end, the safety standard ISO 26262 requires the systematic verification of quality for safety-relevant vehicle software as well as proof that all safety requirements have been implemented - particularly that all functional safety requirements have been correctly and fully implemented.

In order to demonstrate this, the quality of the resulting software must be continually evaluated during the software development process. To do this in practice, diverse quality assurance measures are carried out and applied to the various development artifacts throughout the entire software development cycle. If artifacts are developed iteratively as the range of functions increases, the quality assurance measures will follow and will be applied repeatedly.

Many of these can be largely carried out via an automated process using established processes and tools. However, for the overall assessment of the quality of a software product, there is, in practice, no established procedure. Instead, this is left up to the user.

Project aim

In industry, there were often proprietary solutions, quality surveys, and quality monitoring that were carried out and realized over the course of time, with the aim of obtaining statements about the overall quality via a quantitative evaluation of the existing heterogeneous data. The overall assessment of the quality of a software product occurred via the aggregation of the individual quality measurements. MES developed a generic concept to aggregate quality measurements and implemented them in one tool: the MES Quality Commander.

In the manufacturing sector, the overall quality of a product was already well established. Likewise, the results of various measurement processes were summarized into an overall assessment of product quality. Regularly determining and documenting product quality during the production process was an important way to optimize and control the production processes in order to meet the specifications related to the quality to be achieved.

The aim of the project was to evaluate and systematically expand the procedure for the quantitative evaluation and aggregation of quality measurements developed in the MES Quality Commander, with regard to the procedure’s significance. Additionally, this procedure was compared to the established procedures used in the manufacturing sector, and possible potential for optimization was identified. In order to transfer methods from the manufacturing sector to software development, the similarities and differences between the two domains were worked out and the transfer possibilities were evaluated. The procedures that were applicable to the transfer were compared to the approach implemented in the MES Quality Commander and further development possibilities were derived.