Since adopting Parasoft’s C/C++test tooling, Qinetiq Space has seen a significant increase in software quality.
Qinetiq Space is the Belgian space division of the UK-based Qinetiq corporation. We specialize in the development of high-reliability (hi-rel) products to be deployed in the harsh space environment. Typical hi-rel products are avionics, electric propulsion, microgravity research instruments, robotic systems, space security, space exploration, remote sensing and small satellite systems. We manage the development of these products from concept to in-orbit operations.
To make this possible, the engineers in our Electronics and Software Engineering (ESE) department write safety-critical software applications ranging from boot loaders, low-level device drivers and communication protocols to system services and real-time application software. As the complexity and amount of software in our products continuously increase, so do the risk for system flaws caused by software bugs and the need for early detection of unwanted side effects of new implementations. Consequently, the ability to control software quality was becoming more and more eminent. Fortunately, we made the required investment well ahead in time to prevent loss of product quality and, more importantly, to prevent loss of customer satisfaction.
First time right
The increased risk for failure is not so much driven by the software developers themselves having to write many lines of code but mainly by the difficulty to oversee the interactions in large software systems. Thus, when a bug gets detected, the difficulty and time needed to debug and the associated costs increase with it. The worst case is when the product exhibits a flaw after deployment. Since we deploy in space, where it’s extremely difficult and sometimes impossible to patch software, the long communication distances contribute to the effort and boot loader software is typically programmed in read-only memory. Tools that can assist software development teams in early problem detection and self-verification have therefore become imperative for us to use.
We’ve also learned that it’s advisable to implement some kind of continuous integration (CI) workflow like Subversion and Jenkins. This combo has a low entry level and can be easily integrated into an existing software development process. The application of quality objectives in general, and specifically unit testing, will increase cost and resources during the initial software development, but this far outweighs the cost of bug fixing and time delays during integrated testing and results in efficiency gain during regression testing.
The main objective of our software team is to deliver high-quality, reliable and first-time-right products. We’re required to meet the stringent requirements defined in the European ECSS-E-ST-40C and ECSS-Q-ST-80C software engineering and quality standards. This calls for severe product verification, 100 percent unit test coverage, 100 percent requirements verification coverage, dedicated software safety tests by failure injection and independent software verification and validation.
‘First time right’, in our context, means the product has to safely, robustly and correctly operate according to its specifications in a remote and sometimes inaccessible space environment. It doesn’t mean that software flaws aren’t permitted during development. On the contrary, we apply an evolutional development lifecycle, where at successive project review milestones we achieve and demonstrate increased product quality. By the time the satellite hardware is on top of the launcher, the software has to be ready and validated.
To meet these objectives, our software team relies heavily on the use of hardware simulation. This approach is paramount to quickly achieving reliable results when executing new software on its target processing environment, ie real hardware.
Self-verification
To standardize software engineering methods and be able to quantify software quality through objective metrics, we needed to find an appropriate set of software quality control tools. Our main tool requirements were: easy integration in existing software design environments, the ability to automatically verify MISRA C/C++ coding rules, the ability to select and configure coding rules depending on project needs, the ability to perform unit testing while measuring in parallel the amount of code coverage, the ability to acquire software metrics like code comment density, cyclomatic complexity and structural coverage, and the ability to generate concise software quality reports. The recent drive to implement continuous integration (CI) also requires the tooling to be executed from a script that can be launched from a CI server.
We adopted Parasoft C/C++test to meet all of these requirements. C/C++test integrates all required features in a single application, avoiding the need to have other tools. This is a significant advantage for our software developers since they can use a single user interface to navigate identified issues with just a few mouse clicks. We can also measure modified condition/decision coverage during unit testing, which enormously boosts our confidence in the robustness of our code, and we can adapt coding rules to our specific software quality requirements.
We were able to measure the value of using C/C++test by monitoring the number of software problem reports (SPR) during development. Since we integrated the tooling, we observe a 75 percent reduction of SPRs being issued during software development.
C/C++test has also enabled the development team to self-verify the code quality. Self-verification is an important asset as it reduces the number of bugs that need to be solved after a verification and release cycle. Consequently, the overall development costs have been reduced and no flaws have been observed in our products after deployment. Hence, product quality has increased.
Last but not least, C/C++test has allowed our product assurance manager to acquire objective software process metrics. This has been very instrumental in demonstrating to our customers the evolution of their software product. Thus, the tooling creates customer confidence, improves long-term relationships and empowers future ventures.