High Data Rate Considerations for SDR in Spectrum Monitoring and Recording

April 26, 2021

Story

Spectrum monitoring has become a critical activity for both commercial and defense applications.

Section 1:

An ever-increasing number of technologies are using unprecedented levels of bandwidth. Despite the increase in captured data, it is generally best to collect as large a portion of spectrum as possible. This poses a significant problem for modern spectrum monitoring solutions; analyzing large amounts of spectrum in near real-time is computationally intense. To meet the dynamic capture and processing requirements of spectrum monitoring, software-defined radios (SDR) and external data processing systems have become the de facto standard. High performance spectrum monitoring will require careful consideration of the system architecture to prevent system bottlenecks and to enable efficient data analysis. At a high level, radios will need to connect to the data handling system through a high-speed data link; state-of-the-art SDRs are utilizing radio-to-host connections with speeds up to 4x40 Gbps to fully enable broadband monitoring.

Section 2:

Once the data has been offloaded from the radio to the processing system, there are a variety of bottlenecks that can occur. Ingesting the data through a network interface card (NIC) can cause a variety of problems, the first being dropped packets. Not all NICs are capable of handling multiple Gbps even if they are connected via PCI bus. Once a NIC is overloaded packets will start to be dropped resulting in loss of captured data which is unacceptable in spectrum monitoring applications. FPGA based NICs have been developed to address this issue as they can support higher throughputs. Conventional NICs will transfer packets via a bus to the host controller in a one-to-one manner which can result in congestion in high-throughput instances. If large amounts of data is being ingested using a conventional NIC, packets may be discarded due to its inability to pre-process and aggregate packets. FPGA based NICs can leverage pre-processing and compression to reduce the ingestion workload of downstream processing units such as CPUs and other FPGAs.

Fig 1: Typical ingestion hardware solution

Section 3:

After the data has been ingested by the computing system, it will need to be stored and processed. The architecture of the system should be carefully designed to maximize ingestion rate while minimizing hardware cost. A typical hardware configuration can be seen in Fig 1. Implementing a fan-out storage architecture is ideal for high-throughput spectrum monitoring solutions as it can reduce the performance requirement of individual system components. It is also prudent to have the storage system utilize a ring-buffer to provide the largest collection history while automatically discarding the oldest data. The selection of HDDs versus SSDs for spectrum monitoring storage hardware is both application and cost specific. HDDs have a lower price point and write speeds of approximately 150 MBps while PCI 4.0 SSDs are more expensive but can achieve write speeds up to 5,000 MBps.

Spectrum monitoring solutions with lower data capture and storage requirements can leverage HDDs in RAID arrays. Two HDDs in RAID would support an ingestion rate of approximately 2.4 Gbps. This may seem significant but would only support the continuous capture of approximately 100 MHz of bandwidth assuming that measurement metadata is not stored. Capturing GHz of bandwidth will require the storage write speeds to be orders of magnitude higher as many spectrum monitoring applications currently leverage multiple independent radio receivers to improve performance and capture bandwidth. To meet this increased requirement, NVMe SSD are the best solution for high performance spectrum monitoring storage. A single high performance NVMe SSD could replace a 17 HDD RAID meaning a single device can ingest over 1600 MHz of captured spectrum. While SSDs provide significant performance improvement over their mechanical counterpart, many spectrum monitoring solutions will still require a RAID configuration. To ingest data from a state-of-the-art 4x40 Gbps NICs, a striped array of four high performance SDDs would be required.

In addition to the transfer speed requirements, the computing capability of the system must be able to meet the ingestion and processing requirements. As the amount of spectrum being captured increases, so too does the CPU and processing card capabilities. High capture bandwidths will require multiple CPU cores to be allocated. Storing 160 Gbps of data requires approximately 25 CPU cores to be dedicated to the ingestion process [ntop]. Analysis of this data is suggested to be handled using a distributed computing architecture, onboard FPGAs and GPUs, or some combination. In addition to the CPU cores, several GBs of RAM should be allocated to buffering the data before it is written to the RAID array. For an HDD array the buffer size should be larger to compensate for the write delay but can be reduced in size for SSD-based storage solution.

Fig 2: Packet capture and processing data flow

Section 4:

A significant benefit of SDR based spectrum monitoring solutions is the high level of reconfigurability they offer. Due to the variability of the hardware configuration, the captured metadata is critical in the analysis process. Parameters such as capture bandwidth, carrier frequency, and temperature can vary significantly, and the associated metadata must be stored alongside the spectrum data. The existence of numerous SDR vendors and their unique packet protocols can complicate analysis of the captured data. Utilizing SDRs which are VITA49 compliant will increase both performance and consistency between SDR platform data. VITA49-compliant SDR data will have the captured samples isolated from the metadata. Separation of the metadata reduces data transfers as VITA49 compliant SDRs only send metadata packets when changes are seen by the SDR leaving more room for spectrum capture data. In addition to excellent metadata handling, VITA49 supports high-precision timestamping of the packets with timing correction that compensates for the RF front-end delays resulting in more accurate metadata [IEEE].   

Section 5:

Despite becoming more common across many applications, the majority of SDRs currently available are unable to meet the requirements of high-performance wideband spectrum monitoring applications. In addition to high channel bandwidth, many spectrum monitoring applications require multiple independent receive chains to enable spatial information extraction. To enable high bandwidth capture and processing, the radio and early-stage processing systems need to be closely integrated through a high-speed digital backhaul. The strict integration requirements result in turnkey SDR solutions offering the best performance while decreasing hardware development time. Since data ingestion and processing requirements can be very strict, some turnkey solutions will integrate recording, storage, and playback directly into the solution to ensure optimal performance. It is important to work with vendors that can offer complete solutions when looking at the high-performance spectrum monitoring requirements discussed in this article.