Enabling infrared systems in UASs through SWaP optimization

Story

April 30, 2020

Unmanned aerial system (UAS) platforms continue to reduce their size, weight, and power consumption (SWaP) and enhance their performance and functionality. These improvements are enabled by a new generation of sensing technology; one example is integration of high-performance infrared (IR) imaging systems onto reduced-scale aircraft.

Infrared (IR) systems measure thermal radiation to create images. Taking it further, high-performance IR systems must be capable of measuring the tiniest levels of thermal radiation that are emitted from distant objects; to do so, these systems need to be cooled. This cooling is required to reduce the level of thermal energy that is generated in the IR imager and background, so that the signal received is much higher than the thermally induced background and noise that is generated in the IR detector. Not surprisingly, conventional cryogenic cooling systems can contribute a significant level of SWaP to a UAS. Now, new types of digital readout integrated circuit (DROIC) transduction technology has been developed that minimizes the SWaP of the cryogenics and is driving the reduction of SWaP to the point that cooled IR cameras can be used in progressively smaller UAS.

The improvements in size, weight, and power consumption of UAS has a meaningful impact on the speed and range of the aircraft. SWaP improvements also enable smaller, lower-cost UASs to accommodate high-performance IR cameras that are critical for identification of threats, acquisition of targets, and multi­object tracking.

Along with the obvious UAS trends – shrinking the footprint, enhanced flight time, and additional functionality – major interest exists in UASs that can navigate without human pilot intervention. IR integration is necessary for autonomous operation in order to deal with adverse atmospheric conditions contrary to safe autonomous navigation.

Transduction technology

Advances in transduction technology enable IR imaging systems that have a major effect on UAS capabilities. One factor is the reduction in size and weight of a cooled IR camera system: Smaller, higher-resolution, and higher-dynamic-range focal plane arrays (FPAs) can now be used that produce significantly better images while simplifying the cooling requirements of the system. The improvement in performance of the DROIC, through such technology as advanced sampling algorithms, drives a meaningful reduction in the size and complexity of the cooling system, known as an integrated Dewar cooler assembly (IDCA). A typical payload of a small rotating-wing UAS is less than fifteen pounds, so optimization of the IDCA is an important factor in enabling the use of cooled IR cameras on this class of UAS.

The DROIC itself is a transduction device that, along with a detector, constitute the FPA. Photons radiated or reflected from the target are collected by the detector and converted to electrons. The transduction that occurs is the conversion of thermal energy to electrical energy. The amount of thermal energy that is gathered from a target (as much as several miles away) is very small and is measured in photons. These photons create a proportionate number of electrons in the FPA. The resulting electrons are stored in integrated capacitors in every pixel in the array; a 1,080 by 720 array has almost a million pixels.

The electrical charge stored on these capacitors is converted to a digital code. The infrared image is comprised of the digital code data that is collected across the whole array of pixels (or “frame”) and is transmitted out of the DROIC at a very high frame rate to enable access the image data in real time. This data will be displayed on a monitor to view the image and may also be processed using algorithms to detect targets that may not be obvious to the human eye.

Given that the image data is comprised of a finite number of electrons created by photons emanating from the target, it becomes clearer why cooling is required to reduce the number of thermally generated charge carriers that occur naturally – at any temperature above absolute zero, -273.15 ºC – in the FPA itself. The sensitivity of the imaging system is affected considerably by the suppression of noise in the FPA, in that optimization of noise performance has a direct impact on the ability to extend the distance and altitude that targets can be acquired. Even marginal improvements to FPA specifications such as noise equivalent temperature difference (NETD), a measure of how well very small differences in thermal radiation can be distinguished, can significantly affect the overall performance of the system.

Another major innovation in DROIC technology that is compelling for UASs is the new ICs’ ability to identify and accurately track multiple targets simultaneously. The most advanced devices will provide a global shutter (snapshot) mode with unlimited small windows for multiobject tracking. The size of the windows can be adjusted depending on the situation at hand; for example, an unlimited number of 32 by 32-pixel windows could be tracked at approximately 9,000 frames per second (fps) per window. Keep in mind that a 4K HDR TV operates at 60 fps. The performance of multiobject tracking at high speed is important as small rotary-wing UASs extend their speed well beyond 200 mph and are faced with identifying higher-speed moving targets and threats. Larger UASs, such as fixed-wing reduced-scale aircraft, are also increasing their airspeed capability. When smaller windows are used for tracking, it is also possible to optimize power consumption, which can have an impact on extending the range of the UAS.

Advances in high dynamic range performance of DROICs is another important attribute for UAS IR camera system requirements. A wide dynamic range increases the ability to identify and resolve different objects in a scene that have extremely different thermal characteristics, because the fact is that UASs may be subjected to a number of threats simultaneously. One of these threats may be much larger, hotter, and easier to identify and track than a smaller, less obvious threat that is of comparable concern. A wide dynamic range enables the IR camera system to observe and track both a surface-to-air missile and a sniper at the same time without losing track of the sniper, which would have the more thermally subtle footprint.

Digital readout integrated circuit technology

Let’s look at an example: An up-to-date DROIC is the type of device that is used in advanced infrared imaging systems such as those required to enable small, lightweight, cryogenically cooled UAS systems.

Figure 1 | Block diagram of state-of-the-art DROIC to enable UAS IR camera systems.

The large block in the center (Figure 1) is the active imaging array that consists of 1,280 columns and 720 rows of pixels. Each pixel with its own support circuitry is contained within the 8 µm “pixel pitch.” (Analog circuit blocks are shown in blue and digital circuit blocks are shown in green. Note that the column “ADC [analog-to-­digital] circuitry” is a combination of both.) Early readout ICs used analog outputs only – they were called ROICs rather than DROICs and could not support the benefits that mixed-signal, integrated analog and digital functionality provide. The added digital functionality enables a simpler and higher speed interface to the system as well as better noise performance and lower power consumption.

The digital functionality also enables a range of operating modes for the DROIC. This multimodal operation capability is important as it allows the IR camera system on the UAS to be optimized for different surveillance conditions. For example, “high dynamic range dual integration mode” runs two integration patterns simultaneously on a checkerboard pattern of pixels for detecting threats that have extremely different thermal characteristics. When a target is detected and enhanced sensitivity is required, the signal-to-noise ratio can be improved using “external correlated double sampling” mode along with multiple oversampling methods. The digital functionality also allows range-gated imaging where tracking a long-range moving target through obscurants such as smoke is made possible when coupled with an active illuminator.

Another important capability of this type of DROIC that is important to UAS systems is called windowing. The large (720 p) array is important to provide excellent image resolution, but the whole frame may not be required and it is important to zoom into the detected object, observe it more closely, and track it at higher speed. A window can be created that allows a smaller subset of pixels to be conveyed at extremely high transmission rates. For small windows, a transmission speed of over 36,000 fps is possible on a state-of-the-art DROIC for UAS.

A block diagram of the signal chain that is implemented in the DROIC of Figure 1 is shown in Figure 2. To the left of the vertical dotted line is the active imaging array. The active imaging array itself has a programmable well capacity, global shutter operation, and both an integrate-while-read (IWR) and integrate-then-read (ITR) pixel structure. This setup gives the DROIC a great deal of flexibility about how the information emanating from the target is processed depending on the nature of the target, its distance from the UAS, and how it is to be observed.

Figure 2 | Signal chain of state-of-the-art DROIC to enable UAS IR camera systems.

Each pixel is read out in parallel, row-by-row to a column buffer. The column buffer presents a clean low-noise signal to a programmable gain amplifier (PGA) that sets the polarity and applies gains. Advanced DROICs can use different detector types that can have different polarities (either “N-on-P” or “P-on-N”) known as dual polarity. It is also possible to detect different infrared wavelengths (for example, both midwave and long-wave) using the same DROIC. This situation is known as dual-band and is very useful for enhancing the UAS IR camera system imaging characteristics to create a two-color solution. A practical example of the two-color solution would be to use LWIR for good visibility through poor atmospheric conditions (smoke, weather, turbulence, etc.) and MWIR for better range under good atmospheric conditions.

The PGA outputs are presented to a column-level 14-bit ADC to create digital codes, a configuration known as a column-parallel architecture. Such a column-parallel approach eliminates high-speed internal analog buses, which is used for reducing power consumption and supports very-high-speed digital data transmission. The digital video processor (DVP) performs linearization of the codes and is used to accomplish digital signal processing such as averaging and oversampling. The high-speed serial­izers are used to create streams of video data in a standard format that are then transmitted off-chip on the low voltage differential signaling (LVDS) pins to support very robust high-speed communications that are easy to interface with the DROIC.

Figure 3 shows a part of a newly designed DROIC die to enable UAS imaging systems. The device has been manufactured in CMOS technology but has not yet been hybridized with a detector to create a focal plane array (FPA). After the FPA is constituted by attaching the DROIC to a detector by means of indium bumps, it will be mounted in an IDCA for use at cryogenic temperatures. A portion of the pixel array is shown in the lower right-hand size of the picture. The pixel width is 8 µm; four of these pixels have been magnified and shown in the circular picture insert. The indium bumps are connected to each pixel so that the detector material sits directly on top and delivers photo­current directly into the pixel.

Figure 3 | 8 µm DROIC pixels with connections for indium bumping to detector. Image: Senseeker Engineering.

Future trends

No need for a crystal ball to see that the size, weight, and power consumption of small UAS will continue to reduce while the functionality, speed, and range will increase – this is a given. The more significant evolvement is the ability to add lightweight cutting-­edge infrared imaging functionality that enables a step-function improvement in capabilities and opens the door to autonomous flight. The acronym “DROIC” doesn’t roll off the tongue as sweetly as “CPU” or “GPU,” but in terms of a highly integrated mixed-signal enabling chip, the advances in DROIC technology are just as significant and will be seen shortly on the next generation of UASs equipped with state-of-the-art lightweight IR cameras.

Ross Bannatyne is director of business development for Senseeker Engineering. He previously served as director of marketing for VORAGO Technologies. He was educated at the University of Edinburgh and the University of Texas at Austin. Ross has published a college text called “Using Microprocessors and Microcomputers” and a book on automotive electronics called “Electronic Control Systems” (published by the Society of Automotive Engineers); he also holds patents in failsafe electronic systems and microcontroller development tools. Readers may contact the author at [email protected].

Senseeker Engineering
www.senseeker.com