Designing intelligent video analytics at the edge of IoT
October 26, 2017
Video analytics applications operating within the Internet of Things (IoT) can be improved by using node analytics and logarithmic imagers.
Video analytics applications operating within the Internet of Things (IoT) can be improved by using node analytics and logarithmic imagers. Video analytics applications attempt to take advantage of the abundant information in the everyday world for several reasons. These reasons range from facial recognition to everyday surveillance, but the majority are centered on predictive and behavioral analysis. The information gathered in these applications can be processed extensively at a higher level by cloud computing. However, in-depth processing has its limitations and can be improved in many ways by adding node analytics and logarithmic imagers to the mix.
By adding node analytics to the mix, data analysis can be improved by eliminating communication with the cloud. Cloud computing requires two, if not three orders of a magnitude of more bandwidth than node analytics applications. As a result, node analytics consumes much less computational power and latency is decreased. Densely populated markets, chaotic traffic sections, and city parking spaces are some of the intricate atmospheres that can be sensed for predictive and behavioral analysis using node analytics. High level processing of these environments performed in the cloud can advance business strategies, divert general traffic flow, and improve the efficacy of government-managed parking spaces. However, by implementing lower level software at the sensor node instead of performing analysis in the cloud—latency, bandwidth, security, and power can be improved in these scenarios.
In addition to intelligence at the node, adding logarithmic imagers to the mix can strengthen these systems by providing advantages in areas where conventional imagers fall short. Logarithmic imagers provide a higher dynamic range for image processing, in addition to reduced dependence on luminosity changes. For example, shadows, reflections, abrupt changes in light, and high contrast scenarios are the areas in which a logarithmic imager can outperform a conventional imager. The resolution of these issues in video applications can bolster data capture, which strengthens the analytics at the node. By improving data capture, the overall video analytics application can be significantly improved.
The improvements offered by node analytics and logarithmic imagers can help solve problems in video analytics applications operating within the IoT. Some of the engineering difficulties common to IoT applications are security, decision making latency, data bandwidth, and computational power. These engineering problems are greatly reduced with the elimination of data transmission, which is why at-the-node analytics is attractive for IoT applications. In video analytics applications, limited contrast and dependence on luminosity is a common struggle. This struggle is almost eliminated with logarithmic imagers, which is key for video analytics applications. Overall, video analytics applications operating within IoT are enhanced with node analytics and logarithmic imagers.
Intelligence at the edge
By processing the data based upon expected visual events, the measured data can be transformed quickly into appropriate actions with little or no data communicated to a cloud server. This quick analysis of video data, instead of transmitting to the cloud, localizes the decision making process and improves the latency in the system. The latency in decision making is not only reduced significantly, but the security is enhanced by eliminating the transmission of data that would normally introduce the risk of interception.
Only the most valuable information needs to be connected beyond the node and into the cloud for predictive or behavioral processing. This optimized partitioning of data maximizes cloud value, because full bandwidth of a video analytics frame is typically not needed. Most of the visual data from frame to frame is static on a fixed mounted camera that can be filtered at the node. Edge node video analytics can provide many filtered interpretations to differentiate expected object types: car, truck, bicycle, human, animal, etc. This decimation reduces data bandwidth and associated computational power that otherwise would be required within a cloud server to analyze the full frame rate of video data transmitted downstream. This reduction in bandwidth can be a two or three orders of magnitude improvement over cloud computing applications, which is a key improvement offered by node analytics.
Logarithmic imaging
Video analytics applications can be improved further by addressing common concerns related to conventional imagers with the substitution of logarithmic imaging. Most conventional imagers are linear and use pixels that generate a voltage that is a linear function of light, which can result in a limited contrast. Linear imagers also utilize a uniform exposure phase that limits their dynamic range to the pixel exposure time within the frame rate. Lastly, conventional imager contrast is dependent on luminosity, which can introduce reflection-related contrast issues. These common problems are eliminated with the substitution of a logarithmic imager by using pixels that generate a voltage that is a logarithmic function of light. The differences between a traditional imager and a logarithmic imager can be observed in Figures 1, 2, and 3.
[Figure 1 | Conventional image exposure.]
[Figure 2 | ADIS17002 logarithmic image exposure.]
[Figure 3 | ADIS17002 logarithmic image exposure with built-in edge detection.]
Some conventional imagers struggle with contrast-related difficulties that hinder users from fully capturing their target environment. These contrast issues stem from the linear properties of the voltage generation within each pixel. The voltage that is generated within a linear imaging pixel is directly proportional to the amount of photons striking it; as result, the dynamic range is limited compared to its logarithmic counterpart. The reduced contrast associated with these linear imagers is a result of the reduced dynamic range. This reduced contrast can corrupt analytics at the sensor node in IoT applications, ultimately affecting the overall system performance. The logarithmic imager provides for a wider range of light levels and, thus, increased contrast due to the pixel voltage being logarithmically generated. However, this increased contrast results in higher sensitivity to light, which can be an undesired effect in some applications. Alternatively, this increased sensitivity to luminosity can be an advantage—it all depends on the application.
Video capture with conventional imagers can be further obstructed by reflections in sunny or bright environments. For example, facial recognition in vehicles can become increasingly difficult when reflections are present on the windshield. This obstruction of video capture can negatively impact video analytics by introducing errors in the system or by the loss of crucial data. These reflections are introduced because linear imager contrast between pixels is dependent on luminosity; as a result, reflections are more prominent. This dependence on luminosity can be observed in Equation 1. Alternatively, logarithmic imager contrast is independent of luminosity, due to its natural logarithmic properties, which helps mitigate reflections or abrupt changes in light. The independence of luminosity for logarithmic imagers can be observed by Equation 2.
Beyond individual components
Platform-level solutions help customers rapidly deploy proven intelligent solutions with improved performance at a lower system cost. Analog Devices is partnering up with customers to develop unique, system-level solutions that approach the whole problem. One of these solutions, which is capable of quarter video graphics array (QVGA) imaging analytics, is the ADIS1700x.
[Figure 4 | A functional block diagram]
[Figure 5 | ADIS17002 at angle (left), above at the lens side of the board (center), and back side (right).]
The ADIS1700x is a QVGA analytics imager module in a small form factor with logarithmic sensitivity, combined with digital signal processing that optimizes video performance. The module utilizes a low power Blackfin processor to offer the ability to perform analytics at the node, in addition to an accelerometer for image stabilization, tilt, and impact detection. It also utilizes built-in edge detection for tracking and counting object motion. Each 14 μm × 14 μm pixel has a unique exposure phase, unlike conventional imagers. Conformal coating for outdoor operation make this module suitable for mass deployment that allows the creation of emerging smart city and building applications. The ADIS17001 provides a 110° field of view (FOV) lens, while the ADIS17002 provides a 67° FOV lens. These two options provide for a diverse range of target applications, which include parking spot monitoring, parking violation enforcement, traffic queue detection, and industrial analytics.
Overall, video applications in IoT can be significantly improved with node analytics and logarithmic imagers. Node analytics, instead of cloud computing, can benefit IoT applications moving forward. Logarithmic imagers have benefits that its counterpart cannot match, improving IoT applications even further. In summary, video analytics applications operating within the IoT combined with node analytics and logarithmic imagers makes for a robust, system-level solution.
