Transporting and recording high volumes of sensor data

Story

September 23, 2009

John Wemekamp

Curtiss-Wright

With today's high-volume transmission of data generated by ground- and air-based surveillance radar and various types of sonar, many modern military embedded applications are migrating to 10 GbE.

There are classes of sensors such as air- or ground-based surveillance radars or towed-array or dipping sonars that generate very high volumes of digitized data. This data must often be streamed over long distances to the location of the signal processing equipment. Common solutions today use gigabit signaling of various types. However, as these sensors become faster and more sensitive and thus generate more data, many applications are migrating to 10 Gigabit Ethernet (10 GbE). During development and the continuous evolution of these sensor types, there is a need to accurately record and play back this data in real time.

Data transmission

There are many options available for the transmission of sensor data to signal processing equipment. Open architecture systems designers have generally adopted standards such as 1 GbE or Serial Front Panel Data Port (SFPDP), often used in multiples to achieve the data rates required. For long-distance transmission, standards-based options are available to support single-mode or multi-mode fiber, extending their range to many kilometers if required. 10 GbE becomes an attractive option for long-distance transmission as only a single fiber would be required. A typical mobile, ground-based radar system might deploy one or a number of radar antennas linked to a control shelter located a considerable distance away. This is to provide protection for both equipment and personnel if the radar is attacked by anti-radiation missiles. These would target and destroy the vulnerable emitters but not the shelter. Fixed installations of similar radar systems controlling an entire airspace might also be located many more kilometers from a heavily protected control room.

Many naval applications require physical separation from the severe external environment of, for example, a masthead sensor and the location of the associated signal processing in below-deck equipment rooms. Towed-array and dipping sonars are additional examples of sensors located in severe environments, in this case attached via an umbilical cable to the vessel or helicopter containing the more protected signal processing and display equipment.

Recording for playback

There are many reasons for recording streaming sensor data, including development and validation of signal processing equipment, off-line analysis and threat evaluation, equipment evolution, system testing, training, and live exercise evaluation. Even though the recorded data will have some embedded sensor timing, such as azimuth or elevation for a radar system, fidelity of playback can only be ensured by restoring the accurate timing of the data sampling within the data streams. The best way to achieve this is to time stamp every incoming sample as it is received, which would require a clock source with, typically, sub-100 ns resolution. This allows playback at the original sampling rate and provides the facility to start and stop at selected times within the recorded data.

Disk storage

Newer, high-performance sensor types can typically stream data at up to 800 Mbps. High-speed links from the sensor will commonly use a very lightweight protocol because at these data rates, errors in transmission can only be flagged, not retransmitted. Hence, UDP is used to stream datagrams over 1 and 10 GbE point-to-point links. For recording of any reasonable time duration, terabytes of data storage are required. Just one hour of recording might require up to 4 TB, which would be provided by Fibre Channel rotating disks in either Switched Bunch Of Disks (SBOD) or Redundant Array of Independent Disks (RAID) in benign environments. Because of timing constraints, data is striped onto the arrays without the use of a file system. This also has the advantage that it can be streamed in replay mode to many different computer types as it has no operating system-specific file attributes. Typical of this type of recorder is the SDR1X family (Figure 1), which is offered by Curtiss-Wright Controls Embedded Computing (CWCEC). SDRIX is a rack-mounted system giving more than two hours of 10 GbE recording with options for SFPDP and 1 GbE links to the sensor.

Figure 1: Curtiss-Wright’s SDR1X recorder is a rack-mounted system giving more than two hours of 10 GbE recording with options for SFPDP and 1 GbE links to the sensor.

(Click graphic to zoom by 1.9x)

The latest generations of sensors produce a tremendous volume of raw data, so much so that it would be impossible to record and archive it all. But sometimes the validation of new algorithms, the detection of new hostile capabilities, or the evaluation of a live exercise requires the replay or analysis of many hours of captured raw data. This is only possible with special-to-type recorders able to record and, just as critically, replay this data in true real time.

To learn more, e-mail John at [email protected].