Managing legacy graphics and user interface software

Story

August 11, 2010

Mark Snyder

ALT Software

Graphics represent a challenging area for managing software obsolescence. Most applications that render to the screen or manage a user interface make use of low-level driver calls, such as OpenGL, and extensive User Interface (UI) libraries such as X-Windows or other UI libraries. Deploying these software applications on new platforms when technology insertion is needed because of performance requirements changes or obsolescence is challenging. However, a methodology whereby graphics and UI interfaces can be abstracted without sacrificing performance or flexibility is vastly extending the useful life of legacy software when technology insertion is required.

User interface and graphics software represents a significant proportion of any software system that requires user interactions. In military systems, User Interfaces (UIs) range from real-time embedded displays through extensive digital map and geospatial C4ISR applications. User interface software is unique in that it is highly visible to the computer user while at the same time interfacing directly to graphics hardware through driver calls. Such driver calls include those provided by standards such as OpenGL or by APIs such as rendering libraries provided for a radar processor to draw reticles or text.

In addition, user interface software is highly dependent on operating system constructs such as a windows manager (for example, X-Windows), often also used to provide 2D graphics capabilities. Many times the chief impediment to software portability is in the UI and graphics area. This is even more true when trying to manage legacy software migration to systems that require technology insertion, such as replacement of obsolete graphics chips.

Many tools and methods have been developed during the years to aid in making UI and graphics applications more portable and resistant to obsolescence. For instance, many popular software tools and libraries that abstract the interface to the graphics hardware are available for developers’ use. These tools might employ code generation (example: Presagis VAPS) or pass graphics rendering calls through an abstraction library (example: Nokia QT).

These development tools are very beneficial, but they do not help much with a legacy code base that has extensive dependence on graphics calls. In many cases, technology insertion requires investment in new graphics drivers or window management software to match legacy software’s API needs. The API calls required by the legacy application are implemented in the hardware device driver. In practice, this means that hardware evolution is slowed, and that hardware capabilities do not evolve as they should. For military systems, sadly, it also often means that developers cannot take advantage of new technologies that would greatly benefit the system, such as low-power, low-cost computing platforms that are revolutionizing mobile computing. Industry standard groups such as Khronos (www.khronos.org) have paved the way for new, low-power graphics and UI systems through defining and supporting new standards such as OpenGL ES for embedded graphics. However, these standards are designed to enable UIs and rich graphics experience with drastically reduced APIs and embedded driver sizes. Few would argue that they deliver rich, useful user interfaces, but legacy software cannot take advantage because it was not written to support these new, streamlined APIs.

Simply put, graphics software obsolescence is leaving military systems stuck in the past. This means that soldiers, sailors, and airmen are using computers that cost more and consume more power than they need to, and often deliver substandard operation. In addition, new technologies that make smartphones, TabletPCs, and automobile computers ever more user-friendly and useful cannot find their way to military embedded systems.

New software approaches

Several approaches exist to bring legacy graphics software applications onto these new hardware platforms. One approach that offers much promise is to leverage the concept of virtualization and apply this to graphics and UI systems. Using this approach, graphics-rendering commands – whether legacy APIs, 2D window-management commands, or 3D rendering – can be implemented through a virtualization mechanism similar to those used for other OS and device interfaces. The main challenge in implementing such a scheme lies in preserving the performance and real-time interface to the hardware and graphics resources while still supporting the needed APIs. For this reason, a good virtualization solution must be built from the device-driver level to ensure success. A “hybrid” between a traditional driver and a virtualized one is necessary.

Such a “hybrid” approach can offer high-performance, divergent back-end implementations for new devices in addition to front-end APIs that legacy applications expect. Graphics resources and commands must be marshaled by this layer to deliver real-time performance. The virtualization layer must be prepared to manage graphics resource memory and virtualized device interfaces such as windows. It must also be ready to perform seamless low-overhead API conversions as required.

Embracing the future

The best approach for managing obsolescence in graphics and UI applications is to design and develop for it up front, by choosing a development strategy and tools that attempt to “future proof” an application and by adhering to relevant standards such as those offered by Khronos. When this is not possible, a virtualization strategy, or “hybrid” device driver, can offer a promising alternative. Work remains to be done, but graphics virtualization is a new and promising area. It offers a way forward to bring legacy military graphics systems into the new computing world vigorously charted by smartphones, TabletPCs, and the devices that are changing everyday lives.

Mark Snyder is Vice President of Product Marketing for ALT Software. He has been responsible for many innovations in user interface and real-time computer graphics software architectures and tool chains during his 25-year career working at Quantum3D, Honeywell International, and the U.S. Air Force. He can be contacted at [email protected].