The iPad factor: Preparing for the commoditization of military embedded Virtual Training Simulation devices

Story

June 20, 2011

Andy Morris

Heartwood

The next generation of embedded devices might be less a result of government and industry-led R&D efforts than ever before. These new, consumer-oriented systems will be based on today's tablets, smartphones, and gaming toolkits, offering compelling price points and performance characteristics.

Turnkey embedded military systems today offer extensive integration options and combat-tested performance. However, the typical closed-system approach can limit the full potential of these devices. Recent advances in smartphones, tablets, and gaming tookit software and hardware technology now deliver visual fidelity and performance previously limited to desktop PCs. Imagine a pilot being able to troubleshoot an in-flight emergency, like a landing gear deployment problem, with the same visual troubleshooting procedure presented in classroom training. This capability is available today for those who choose to invest in the new generation of Virtual Training Simulation (VTS) tools and techniques.

There are two primary factors driving development in VTS for embedded military systems: the commoditization of high-performance graphics on mobile devices, and royalty-free or open-source game toolkits. As an example, the U.S. Army is testing the Joint Battle Command-Platform smartphone based on Google’s free Android operating system for field operations, including platoon location via GPS. And the open-source Delta3D engine was used to prototype the Nemesis game-based-learning application, with an objective to train operators in the decision-making and psycho-motor skills required for the U.S. Army Humanitarian Demining Program.

The Army’s Connecting Soldiers to Digital Applications (CSDA) program is also sponsoring the development of mobile simulation applications. Miniaturization and economies of scale have brought affordable realism to tablets, while leading game toolkits now allow developers to deploy the same source code from dedicated game consoles down to Android and Apple iOS devices. Agile organizations will continue to take advantage of both hardware and software advances in the tablet/smartphone and game toolkit realms to quickly and affordably build highly deployable solutions.

The tablet as a Just-In-Time (JIT) training platform

With fast, dual-core CPUs and independent graphics processors, the current generation of tablets enables the simulation of highly complex and detailed procedures, such as diagnosing fault codes from an onboard weapons system to determine the appropriate tools and techniques required for repair. The original Apple iPad included a single 1 GHz A4 processor and a PowerVR SGX 535 Graphics Processing Unit (GPU). This was indeed a powerful first-generation offering, but limited therealism of training simulations by reducing the speed of draw-calls far beneath graphics cards found on entry-level PCs. The iPad 2 now contains a dual-core 1 GHz Apple A5 processor (see Figure 1) as well as updated dual-core PowerVR SGX543MP2 graphics. This allows for 3D performance up to nine times faster[1]. Technology research firm AnandTech.com found a more modest but still impressive range of 3 to 7x in GLBenchmark 2.0 improvement over the original iPad, running games that had yet to be optimized for the enhanced iPad 2 capabilities[2]. And the expansion of graphics powerhouse NVIDIA into the mobile market promises to maintain healthy price and performance competition in the market.

Figure 1: Original iPad versus iPad 2 GLBenchmark 2.0. iPad 2 allows for 3D performance up to nine times faster. Data courtesy of AnandTech.com

(Click graphic to zoom by 1.9x)

Complex maintenance and troubleshooting procedures that were recently limited to paper can now be rendered effectively on a compact display. Even laptops are being made obsolete, or at least presented with a compelling trade off, by these new ultra-portable computers that offer near-laptop performance for a fraction of the size and weight. And a networked tablet turns an Interactive Electronic Technical Manual (IETM) into a document that can be updated with the latest technical updates wherever the warfighter needs operational support. High-resolution displays (up to 960 x 640 pixels) make mobile simulation a possibility on once unthinkably small screens like the 3.2-inch diagonally HTC Aria or the 16:9 aspect ratio Motorola Droid. And standards bodies such as S1000D and DITA remain vested in making sure that their documents are accepted by procurers of embedded military systems.

Game toolkits enable software advances

Toolkit providers now recognize the opportunity to broaden the customer base for development beyond recreational gaming. A driving factor is the Armed Forces’ adoption of virtual training and simulation as key tools for increasing training capability and capacity, while lowering training costs and hazards associated with live training. According to research from Visiongain, global spending in 2009 on military simulation and virtual training reached $8.4 billion, and will continue to increase through 2020[3].

As promising as commercial tablet development is to embedded military systems, recent developments in software are even more compelling. The commercial video game industry is responsible for driving DirectX and OpenGL graphics performance on PCs. The defense industry recognized this fact several years ago, and has been leveraging the same powerful toolkits to build “Serious Games” (SGs) such as America’s Army and simulation-based training. However, until recently these toolkits suffered from significant disincentives: They cost hundreds of thousands of dollars or incurred large royalty payments, and development had to be targeted to one specific platform such as a PC game console.

Now, a developer can produce VTSs for PC instructor-led training, and deploy the same code base to an embedded military mobile device running Android or Apple iOS for little or no additional cost. This approach is opening up game toolkits to hundreds of new developers, many of whom contribute valuable input that serves to improve the performance and feature set. Expanded capabilities, such as the bundling of expensive physics libraries, bring even more developers onboard who continue to push the envelope and pioneer new use cases in a virtuous circle.

One example is a commercial game toolkit adopted to demonstrate a military training concept: The company created a VTS prototype that requires users to perform a routine maintenance task from a first-person perspective. Individual steps within the maintenance trainer require learners to open hatches, move levers, toggle dials, and use a grounding tool. Whenever learners miss a step or perform a step out of order, a warning appears, explaining the error and its consequences. This critical-path training approach ensures that learners master proper operations and maintenance procedures before working with actual military equipment. Once the learners complete the maintenance task, a series of questions provides a measurable check-for-understanding.

This application was developed by Heartwood in collaboration with Unity 3D, and Raytheon leveraged this unique software technology when it selected Heartwood to develop a fully immersive 3D maintenance training program for the Patriot Air and Missile Defense System’s radar set (Figure 2). After a PC version was completed, the client requested a Web-enabled method of user access, which resulted in a full Web-enabled virtual training program. These applications are now being deployed to the Apple iPad, using the same Unity 3D code base but optimized for JIT or refresher training. Additional optimization was required, but the core functionality remains intact as well as the original 3D models, simulation, and logic. Carrying the evolution another step forward, these fully mobile versions of the application aim to deliver the ultimate promise of embedded military systems: sustainable, on-demand decision support.

Figure 2: Interactive Patriot Missile training on the Apple iPad

(Click graphic to zoom by 1.9x)

Transforming commoditization into strategic advantage

Advances in tablet computing, smartphones, and game development have the ability to transform embedded military systems. Additionally, off-the-shelf hardware and low-cost toolkits make VTS a reality in an aircraft cockpit, ground vehicle MFD, or anywhere the warfighter needs to make the right decision the first time. Prototypes can be built quickly and effectively to determine whether the cost and performance advantages merit supplementing or replacing existing embedded technologies.

References:

“The iPad 2’s Processor and Graphics Are Opening Doors,” by Matthew Murray, PC Magazine, March 3, 2011, www.pcmag.com/article2/0,2817,2381348,00.asp
“Apple iPad 2 GPU Performance Explored: PowerVR SGX543MP2 Benchmarked,” by Anand Lal Shimpi, AnandTech.com, March 12, 2011, www.anandtech.com/show/4216/apple-ipad-2-gpu-performance-explored-powervr-sgx543mp2-benchmarked/2
“The Military Simulation and Virtual Training Market 2010-2020,” Visiongain report, August, 2010, www.reportlinker.com/p096388/The-Military-Simulation-and-Virtual-Training-Market-Analysis.html

Andy Morris is a simulation technology consultant who has pioneered groundbreaking projects with Lockheed Martin, Boeing, Northrop Grumman, Sikorsky, and FlightSafety International. Successful applications include computer-based operations and repair training for the F-16 and F-35 multirole fighters and an innovative toolkit for building immersive classroom pilot and maintenance training. Contact him at [email protected].

Heartwood 888-781-0274 www.hwd3d.com