本文章目前仅提供英文版本。
Software Quality Monitoring Starts with Quality Assurance Reports
It's important for all of us to keep an eye on the quality of our software projects. Even for smaller projects, it is no longer possible to have a comprehensive overview without tool support. Where do we get all the data we need to assess the quality of our projects? The most important source of metrics in our projects are the various quality assurance activities we perform according to our defined process alongside many other important tasks. In each of the validation steps we generate test reports to document the results of our activity. These test reports contain the necessary information to evaluate the quality of our project.
Let’s look at typical dynamic testing examples
When customers start with software quality monitoring, they always look at the results of dynamic testing.
The most commonly used dynamic testing tools in model-based development (MbD) are TPT, BTC Embedded Tester, and Simulink Test.
To get an idea of the number of reports we generate in our projects, let's do a sample calculation.
Each time you run a test on a model, one report is generated. If you run your tests only once a week (hopefully more often) and have 43 models, you'll need to access and review data from 860 reports each month. For a typical project, this number is even higher.
For the other validation activities we get similar numbers. A huge amount of reports and an even larger amount of data.
What about static testing?
Static testing is another significant activity in quality assurance and provides important metrics to validate your implementation. It includes guideline checking to ensure the system conforms to predefined standards (MISRA SL/SF, MAB, Functional Safety, …) and best practices and assessment of the architecture and complexity of the model.
Common tools used in static testing are MES Model Examiner® (MXAM) and Model Advisor.
What other metrics do we see in the projects?
Once dynamic and static testing has been covered, projects look at issue tracking next:
- How many issues are currently open?
- How many new issues have been reported?
- How many issues have been accepted?
- How many issues have been closed?
- And so on.
Projects do not use common formats for reporting issues. Common issue tracking tools are JIRA, Redmine, the issue tracker in the version control system (Github/GitLab/...), and many more.
How do we access these metrics? To access these metrics, we utilize the APIs of the respective platforms, which allow us to retrieve issue tracking data in real time. Consolidating this data through API connections provides us with a unified view of project progress, enabling easier monitoring and management of issues across different platforms.
After that, projects start to look at the quality of requirements. This is a broad topic in itself.
What about code-based development?
The methodology is the same – only the tool reports are different.
- For dynamic testing we see TESSY, vTESTstudio, Cantata and many more.
- For static testing you can use Astreé, QAC, PolySpace, PC-lint and many more.
If you would like to learn more about the numbers we use from these test reports, please feel free to reach out to me.
Effective tool support for your software quality monitoring
An effective way to monitor quality is to use the MES Quality Commander® (MQC). This tool collects important data throughout the software life cycle and evaluates key performance indicators such as guidelines, complexity, test results, coverage, and review results.
MQC works well with many development tools, such as TPT and MXAM, to provide a comprehensive overview of the entire project including dynamic and static testing. These integrations help you turn your data into useful insights to ensure continuous improvement and maintain high quality standards in your projects.
Learn more
Interested in learning more about software quality monitoring? Then visit our blog for further insights and best practices on these topics.
