The evolving role of GPGPUs in critical military intelligence

Story

June 12, 2017

Doug Patterson

Aitech

For office workers, staring at a screen for hours a day can seem mundane, creating a feeling of tiredness or boredom. They escape by taking a break, walking away, and coming back in a few minutes after resting their eyes. Think that would fly in a military surveillance environment, where personnel must continually scan images to identify potential threats or persons of interest? Definitely not.

Mundane process or not, managing data in military environments is not an exercise to be taken lightly. Thankfully, computing technology has taken over the heavy lifting of this security task. However, the increased use of displays and electronics has created a unique situation that could actually impair the system’s ability to perform the needed data computation in a timely and safe fashion.

Better images mean more data

From situational displays to tactical ones, the number of panels being used in military and defense applications is growing exponentially. This equates to more data and images that need to be managed. Not only that, the increasing quality of images, which translates to more pixels and higher resolution, leads to a larger quantity of data that these systems need to handle.

How to properly handle this increasing amount of information and make it effective is an important consideration in critical object detection, classification, and threat assessment throughout military and defense applications.

As data requirements grow, critical questions regarding the ability of embedded systems to quickly and efficiently process the information, and then properly identify objects, come into play (Figure 1):

Objects are often partially obscured or occluded from visual imaging, both on purpose and unintentionally – how does a system “fill in the blanks” to ensure that it can accurately classify an image and make the correct threat determination?
Environmental factors, such as fog, rain, and adverse lighting (such as at dusk or with a lens bloom) can affect image recognition – how does a system counteract these unavoidable situations to ensure images are being registered appropriately?
Enemy combatants (or, conversely, pedestrians) can be improperly classified if data is slow or of a low quality – how can a system confidently auto-identify persons of interest and recommend suggested actions?

Figure 1: Proper identification of enemy combatants relies on critical data processing that can discern images and recognize hidden objects, even in adverse lighting conditions.

(Click graphic to zoom by 1.9x)

The struggle to balance processing

The computing systems of today are struggling to keep up with the growing amounts of data that needs to be processed, while still making accurate connections between images, threats, and classifications.

At the heart of the system is the CPU, whose cores are being overburdened with running at a very high rate to process image data while performing task-oriented functions. The constant increase in applications and features just keeps adding to the CPU load. The local cache and main memory are pushed to the limit, with a critical bottleneck looming within the embedded system.

This reality means that systems are limiting computing power and being forced to use lower resolution images as well as lower frame-rate cameras. Instead of employing best-in-class electronics, designers are dumbing down the sensors, pulling back on functionality throughout the system or even making data inputs smaller to keep up with the compute environment and avoid overloading the system.

These should not be areas of compromise, since lower quality of images and fewer frames per second typically lead to inaccurate identification and classification of objects. Threats that could have been easily identified using better-quality images may go undetected. These limitations mean a slower response and information that is not in real time.

Do we need to compromise?

GPUs that have been adopted from the graphics-intense gaming industry have proven to be an integral method of offloading the high data- and image-processing requirements of today’s military systems. They balance system requirements and free up the CPU to perform operational functions, instead of processing all data points that come through the system.

Even though they have been around for a while, GPUs are just now gaining traction in many practical military applications that use modern, rugged embedded systems, especially with the growing need for real-time situational awareness on the battlefield. To understand why GPUs are finally being considered in military applications, let’s think about older display technologies.

If a diagonal line was drawn, and was pixelated, the system blurred the line to straighten it, but that used a significant amount of CPU time. For example, image gradients and shading are being handled by the GPU instead of the system’s CPU. The GPU handles this processing in the background, thereby freeing up the CPU to handle operational tasks and balancing the overall system load.

Differences in processing function

A CPU is structured with a few complex cores and uses serial processing, whereas the GPU uses parallel processing and many, many smaller cores (numbering in the hundreds) that can be rearchitected to process multiple data elements simultaneously. There is no bottleneck.

If we are using only a CPU as the main computing engine, eventually it will choke. By pushing some of the computation-intensive portions of an application to the GPU, while the rest run on the CPU, system load is far more balanced.

Because it has evolved into an extremely flexible and powerful processor the GPU is capable of handling certain processes faster than the CPU due to:

Programmability
Precision (floating-point operations)
Performance – numerous, smaller cores to process parallel workloads

In the computer industry, GPUs are used not only for graphics rendering but also in game physics calculations (physical effects such as debris, smoke, fire, and fluids). This model has also been used to accelerate nongraphical applications in computational applications that use mathematical algorithms.

CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface (API) model created by NVIDIA that addresses complex computing problems. (Figure 2.) So, the faster the CPU and GPU work in tandem, the better the computing performance.

Figure 2: Example of the CUDA processing flow. Graphic courtesy of Aitech/Tosaka – CC by 3.0

(Click graphic to zoom)

It enables software developers to use a CUDA-enabled GPU for general-purpose processing – an approach known as GPGPU [for “general-purpose graphics processing unit].

These rugged GPGPU systems used for military applications are packing high power-to-performance ratios into compact, rugged form factors that can offer as much as 1 TFLOP of processing with only a 7.5 W draw, in some instances. One example: Aitech’s next generation A176 combines NVIDIA’s Jetson TX2 along with a number of I/O options and an internal SATA SSD with secure/quick erase that is aimed at in-the-field, in-flight, and on-the-move military applications (Figure 3).

Figure 3: Compact, high-performance, rugged GPGPU systems are providing increased intelligence to military personnel for situational awareness on the battlefield.

(Click graphic to zoom by 1.9x)

Deep learning and neural networks

The evolution of using GPGPUs in military applications takes the form of deep learning, of which CUDA is an integral part. Specifically, deep learning helps the embedded system operate as the brain does, making “neural” connections that apply learned inferences to image recognition and data-processing requirements. The decisions the system makes are based on learned actions developed over time, where various inputs have been prioritized and fed into the system, much like neurons prioritize information fed into a human brain.

In essence, the system can self-adapt to proceed in logical steps that help increase object recognition and threat detection.

GPGPUs offer the user many advantages, from handling increased graphics output to operating as DSP engines, taking on the cumbersome task of image characterization as well as identification. This continuous learning process provides a solid output for the system to act on. The system thinks: Have we identified a friend or foe, and what actions need to be taken?

Working like the brain, this type of processing uses sophisticated pattern recognition in a multilayered, neural network approach. It enhances system intelligence and efficiency by allowing images to be identified more easily, while assigning context to the image itself for an even deeper understanding.

Take an image of car, for example: The computer processes what it understands, then parses the information down and looks for known patterns.

As it works through the computations, it rearranges data and then weights each attribute based on importance, so it can identify the image, whittling down the options using learned decisions. Each node in the system looks at multiple inputs and feeds its own output to several other nodes:

1. A car rides on the ground, therefore this is not an aircraft (image in context).

2. A car has four wheels, therefore this is not a motorcycle or tank (pattern recognition).

3. A car has a closed structure, therefore it is not a truck (discerning specific attributes).

4. And so on, until the system identifies the make, model, and even color of the vehicle.

The key: The learned intelligence the system gains from the exceptional processing speed enabled by the GPGPU.

Recognizing what’s to come

As military applications use more displays, and those displays provide more data, which is then continually fed into embedded systems, the need to properly manage this information is only going to increase.

Today’s military services are demanding the highest performance in rugged systems. Multicore GPGPU systems fit this model with higher data-processing capabilities, a parallel computing architecture, and better handling of complex computations that provide increased system intelligence for more intuitive feedback.

Doug Patterson is vice president of the military and aerospace business sector for Aitech Defense Systems, Inc. He has more than 25 years of experience in marketing and business development as well as product management in telecommunications and harsh-environment electronics. He served on VITA’s board of directors and holds three patents in advanced metered mailing systems and nonvolatile memory redundancy mapping. Doug holds a BSEE from BEI/Sacred Heart University. He can be contacted at [email protected].

Aitech www.rugged.com