Can Agile environments accelerate embedded software verification and certification?
BlogJanuary 27, 2015
Jay Thomas
LDRA Technology
CODE QUALITY BLOG: Although Agile development is being adopted by most software development organizations, most embedded developers, especially those seeking certification, continue to resist using Agile methodologies. Developers perceive more risks with an iterative Agile approach versus the traditionally regimented methodologies and artifacts required to achieve certification. Specific concerns stem from how requirements are captured in an Agile process, and how the test early-and-often principles of Agile development can be fulfilled prior to the embedded systems hardware being available.
What the naysayers aren’t taking into account is the range of tools already available that help ensure that the objectives of certification are met while adopting Agile methodologies. What knits it all together? Requirements traceability enables software analysis and verification in every sprint and hardware simulation tools make it possible to continuously verify. Finally, workflow management tools help to bring all project artifacts together into a single collaborative environment along with defining and managing project certification standards objectives.
Capturing user stories
So, how do you get the rigor? Opponents claim Agile processes eschew the formal requirements that all certification activities hinge on. Although development on an Agile process starts before a complete set of functional requirements are available, it doesn’t mean that the requirements generated during the Agile process are any less rigorous than traditional regimented methodologies.
Implicit to the Agile approach is the concept of fail early and often, starting with the system requirements. The up-front planning in Agile requires working with customers to develop a series of user “stories” that encapsulate the desired system functionality. The order in which these stories are addressed by each iteration of the development process is then defined, with the objective of each iteration being a release that it is completely functional, fully tested and a potentially deployable system containing the highest priority features.
Captured as either a document, or in a requirements capture tool, user stories can be brought together within a workflow management tool such as LDRA’s TBmanager. As code is developed, an entry can then be submitted to the tool, enabling the code to be traced back to the user stories. Downstream, the workflow manager can also map the code to the software verification activities and results. The workflow manager then becomes the seat of all verification evidence required for certification.
Building quality into software
Producing potentially deployable software for systems that require certification requires that all of the certification standard verification objectives for the software be met, a perfect complement to the Agile mantra “fail early and often.” The highest level of software quality is achieved via a combination of static and dynamic software analysis techniques, all of which can be accessed from within the workflow manager to ensure that the results and context of the analyses are maintained.
Static analysis
Static analysis refers to the practice of reviewing the code under development with the objective of finding and fixing quality issues without executing the code. You use static analysis, for example, when you use an automated tool to verify code standards compliance. Certification standards require the use of coding standards and quality analysis to verify that developed code has been uniformly designed and implemented. The enforcement of these standards throughout all phases of development, integration, testing, and validation provide several advantages:
• Latent defect removal
• Improved code readability and maintainability by creating uniform code
• Prevention of overly complex code which is more prone to error and harder to maintain
• Identification of code that is either unreachable or makes test construction for code coverage challenging
• Production of more modularized code that is easier to trace to low-level requirements
Dynamic analysis
Agile methods use testing to provide feedback on an ongoing basis about how well the emerging product meets business needs. Agile teams test continuously as it’s the only way to be sure that each iteration’s features have been done and progress has been made.
For embedded systems, software development hinges on the availability of a target platform, but that hardware is typically not available until late in the development lifecycle. Agile developers depend on hardware system simulators, such as Wind River Systems Simics, to help plug this gap.
These tools simulate the full target system and can run unmodified target software (same boot loader, BIOS, firmware, real-time operating system, board support package (BSP), middleware, and application) while inside the simulation framework. Using a hardware simulator means that the hardware dependent testing on an Agile project can start in the earliest possible iteration.
Complementing this, test automation tools can automatically generate and execute test cases on the simulated hardware (Figure 1). Automated test case generation and execution can be run often, providing feedback in minutes. The test case generation, execution, results, and status of these tests can then be controlled from within the workflow manager to provide visibility into the progress of the current iteration.
For certification, it is necessary to measure test completeness using code coverage. Certification requires an appropriate level of testing rigor, meaning that all testing must be requirements-based and performed at the system level. The feedback, knowledge, and understanding required to improve test effectiveness is simply not possible without code coverage analysis, and it provides an additional measure of assurance that the potentially deployable system objective for current iteration is being met.
Certification and documentation
Using a workflow manager as the host for the tools used and results generated throughout the development process makes generating the documentation required for certification extremely straightforward. All of the project artifacts can be accessed from the tool, assisting in the preparation of data to present to the certification authorities.
This also is key to closing the loop on accelerating Agile methodologies with embedded systems development for projects seeking certification. Managing the project artifacts within a workflow manager helps ensure that from requirements through host- and target-based verification results, they are all maintained throughout each iteration of the Agile project.
Read more Code Quality Blogs here: Is code coverage analysis necessary? and Is 100 percent code coverage analysis essential?
