What use are test tools for applications based on software reuse?

Story

October 10, 2012

Mark Pitchford

LDRA Technology

The use of test tools is vast across countless applications, but of particular value when used for quality assurance in the customer environment.

When it comes to military software technology development, the software development paradigm is very different from that of, say, automotive technologies. In military technology, duplicate applications and systems are the exception, not the rule. Yet, if there were a way to adapt software test processes to maximize code reuse in military applications, the certification process could be simplified and the software could be reused effectively, making development faster and more economical. Even better, it has the potential to bring the benefits of increased confidence through use of software modules already qualified for a previous application, rather than on the basis of the sheer weight of numbers of a family sedan’s production run.

A look back influences the future

Although the aforementioned approach has merit, issues remain. For instance, it assumes that when we set out to reuse source code, the code is sound because the application has been proven in the field. But what if some of the new functionality builds on paths through the code that in practice were rarely or never exercised? Even well proven code is likely to now be handling very different data. How will it behave?

Dipping into the toolbox of modern software testing tools can help to answer these questions and ensure that code is robust, despite the varying demands of an endless series of different applications.

Anyone who has read of the Ariane 5 failure on June 4, 1996 knows the dangers inherent in any false assumptions. Ariane 5 failed because of a software exception raised in the Inertial Reference System – even though the design was almost exactly the same as that used successfully on the Ariane 4, particularly in the case of the software [1].

If the Ariane 4 Inertial Reference System source code had been subject to structural coverage analysis, all the relevant paths through the code would have been proven to behave in a robust manner. The use of appropriate boundary cases to show robustness in extremis would have shown there to be an unprotected data conversion from 64-bit floating point to a 16-bit signed integer value. At the time, that would probably have appeared pedantic and irrelevant from a developer’s perspective – something that could never cause a problem with the Ariane 4. But it was relevant to the Ariane 5.

Fast forward to structural coverage analysis

Move on 16 years and these structural coverage analysis principles have been embraced by the best test tool suites not only in dynamic analysis, but they have also been automated across the entire scope of software development. Requirements traceability tools, for example, provide a traceability matrix that is permanently up to date and relevant throughout the application’s development life cycle.

Where a new custom application is being developed from an existing one, tools can alert developers when source code can potentially be affected by the revised requirements. In Ariane 5’s case, such a capability could have highlighted the need to retest the Inertial Reference System. It might even have had relevance at the design phase, when it could have allowed a comparison of the overhead implied by different approaches to meeting each revised requirement.

Static analysis tools not only confirm that source code meets the coding standards in force at the time of writing, but they can also analyze the code from the perspective of a revised standard at the time of reuse. Dynamic testing proves the capability of reused code in extremis at the time of writing, and facilitates automated regression testing to show that any enhancements for the latest project have not compromised the previously proven functional capability and robustness.

Test tools and software reuse – the perfect match

Test tools are useful for far more than custom developments, but as these examples show, the customer environment is arguably the one to which they bring the biggest quality assurance benefits.

Reference:

1. www.di.unito.it/~damiani/ariane5rep.html “Ariane 5 Flight 501 Failure – Report by the Inquiry board”

Mark Pitchford is a Field Applications Engineer with LDRA Ltd. and has more than 25 years’ experience in software development, working in industrial and commercial project development and management in the UK and internationally.