How Open Acceleration Standards are Driving Safety-Critical Development
July 14, 2022
Story
The need for graphics and compute acceleration in embedded markets is growing. Cameras and sensor arrays are increasingly central to many use cases in diverse industries ranging from automotive to industrial and are generating increasingly rich data streams that require sophisticated processing.
At the same time, advanced user interfaces are being developed using high-quality 3D graphics and even augmented reality technology. However, the need to deploy accelerated processing, combined with the complexities of safety-critical certification, has created a confusing landscape of processors, accelerators, compilers, APIs, and libraries, driving up integration costs for embedded accelerators, which in turn has constrained innovation and time-to-market efficiencies.
Open standards have an important role in helping hardware and software vendors successfully navigate this complex technology environment. Acceleration standards for the embedded market can enable cross-platform software reusability, decouple software and hardware development for easier deployment and integration of new components, provide cross-generation reusability, and facilitate field upgradability. Such standards reduce costs, shorten time to market, and lower the barriers to using advanced techniques such as inferencing and vision acceleration in compelling real-world products.
Khronos Acceleration Standards
For over 20 years, Khronos has created open, royalty-free API standards that enable software applications libraries and engines to harness the power of silicon acceleration for demanding use cases such as 3D graphics, augmented and virtual reality, parallel computation, vision processing, and inferencing.
Khronos open, royalty-free, interoperability standards
Many industry experts agree that we are facing the end of Moore’s Law, and so there is increasing industry interest in parallel computation that shifts the focus of acceleration from frequency scaling of single cores to processors wielding large arrays of cores that can be leveraged to significantly accelerate data-parallel algorithms.
Broadly, Khronos acceleration standards for parallel computation can be divided into two groups. High-level programming frameworks, such as SYCL and OpenVX, focus on streamlined development with effective performance portability across multiple hardware architectures. By contrast, low-level APIs such as OpenCL and Vulkan provide direct, explicit access to hardware resources for maximum flexibility and control, using the SPIR-V intermediate representation for kernel and shaders programs.
The Khronos family of compute acceleration standards
SYCL (pronounced ‘sickle’), enables code for heterogeneous processors to be written together with the host application code in a “single-source” file using standard C++ for portable acceleration on a wide range of hardware. SYCL will typically compile the part of the C++ application to be offloaded onto an acceleration processor into a lower-level API such as OpenCL.
OpenVX (VX stands for ‘vision acceleration’) provides a graph-level abstraction to construct imaging and inferencing functionality through connecting a set of functions or ‘Nodes.’ This high level of abstraction enables silicon vendors to effectively optimize their OpenVX drivers for efficient execution on almost any processor architecture. OpenVX graphs can combine vision processing and neural network nodes for global optimization and can directly ingest complete trained networks in Khronos’ NNEF format, which provides a stable specification with open-source converters from diverse neural network frameworks.
Vulkan is the only open standard for high-efficiency, cross-platform access to modern GPUs. Vulkan can accelerate compute operations on any compatible GPU using compute shaders, alongside graphics rendering. Vulkan brings a host of benefits to developers of next-generation 3D graphics and compute including simpler drivers, direct and explicit GPU control, multiple graphics, command and DMA queues for multi-threaded applications, and offline front-end compilers for multiple shading languages.
OpenCL is a low-level standard for cross-platform, parallel programming of diverse heterogeneous processors found in PC, servers, mobile devices, and embedded devices, including GPUs, DSPs, and FPGAs. OpenCL provides C and C++ based languages for programming kernel programs that can be compiled and executed in parallel across any processors in a system, with explicit control over which kernels are executed on which processors. OpenCL complements GPU-only APIs by providing a simpler programming model, relatively lightweight run-times, increased language flexibility, and more tightly defined precision for compute operations.
APIs Designed to Streamline Functional Safety Certification
Safety certification is essential in many markets such as automotive, autonomy, medical imaging, transportation, industrial, energy, and avionics. Systems where a compute or display subsystem failure would pose a significant safety risk require system-level functional safety certification and compliance with safety standards such as RTCA DO-178C Level A / EASA ED-12C Level A (avionics), ISO 26262 ASIL D (automotive), IEC 61508 (industrial), and IEC 62304 (Medical), as well as adherence to guidelines from industry groups such as MISRA and AUTOSAR.
The demand for advanced accelerated graphics and compute is growing in these markets, and these accelerators must be included in system-level safety certifications. The high cost of safety certification is motivating the development of open standard acceleration APIs that are designed to streamline the system-level certification process by reducing API surface area through eliminating functionality that is unnecessary for embedded markets, by increasing run-time determinism, and by providing robust error handling.
Khronos has a 20-year history of adapting mainstream acceleration APIs for safety-critical markets, starting with OpenGL SC 1.0 in 2003. These initiatives leverage proven APIs with multiple shipping silicon implementations and that are already familiar to developers.
The history of Khronos safety-critical 3D APIs
The recently announced Vulkan SC 1.0 streamlines the Vulkan SC 1.2 API and increases run-time determinism. Vulkan SC pipelines can be compiled offline and statically analyzed to understand data flow and memory usage. The memory needed for pipeline execution can then be reserved at device creation time as fixed-size pools to both minimize overall memory usage and avoid the need for runtime memory allocation. Vulkan SC enables system implementers deploying GPU-accelerated graphics and compute to meet safety-critical obligations and provide certification evidence packages with reduced cost and effort. Vulkan SC can also be invaluable for real-time embedded applications, even if not formally safety certified.
OpenVX has a safety-critical profile that enables the rapid deployment of trained neural network models, using OpenVX as an inferencing engine that is ideal for safety-critical markets. The OpenVX safety-critical profile minimizes run-time API surface area and run-time size by separating the development of OpenVX graphs from their execution by enabling the run-time to ingest a binary graph definition that has been pre-compiled offline. The OpenVX specification has also been annotated with functional requirement tag numbers and OpenVX headers are MISRA-C compliant.
Khronos has also recently formed the SYCL Safety-Critical Exploratory Forum to investigate industry requirements for a general parallel programming API for accelerated compute using SYCL’s standard C++ single source programming model in safety-critical markets. Hardware-centric API designs are increasingly converging with sophisticated compiler technology, bringing significant new levels of programmable flexibility and portability when tapping into silicon acceleration for compute acceleration. The SYCL SC Exploratory Forum is in the discovery phase to fully understand and build consensus on industry requirements before starting standardization efforts. Any company is invited to join the Exploratory Forum for at no cost, and with no IP licensing obligations.
Standards Organizations Working Together
There are many standards development organizations (SDOs) in the industry each with its own area of expertise, making it important for SDOs to have productive liaison relationships to avoid duplicated effort and to ensure diverse standards from different organizations work well together.
Khronos has recently announced a liaison with AUTOSAR, the worldwide development partnership of vehicle manufacturers, suppliers, service providers and companies from the automotive electronics, semiconductor, and software industry. This new cooperative agreement encourages a productive flow of information while respecting the confidentiality and IP Framework of both organizations. It enables Khronos members to receive information and insights about AUTOSAR activities and vice versa. The end goal is for AUTOSAR use cases and requirements guide and influence the evolution of Khronos standards and to enable the AUTOSAR platform to effectively utilize those Khronos APIs.
Khronos has also worked closely with the European Machine Vision Association (EMVA) over the last two years to establish the Khronos Camera Working Group, which is now building a new open standard API for controlling embedded cameras.
The development of sophisticated optical systems, image sensors, and vision processors is accelerating, and embedded systems often tightly integrate camera sensors with image, vision, and inferencing accelerators in self-contained systems. While the Khronos APIs outlined above can be used for vision and inferencing acceleration, there is not a widely adopted, open standard, cross-vendor API to ease software integration of new camera subsystems, resulting in unnecessarily high integration costs for camera technologies.
The need for an open standard Camera API
The Camera API that Khronos and EMVA members are designing will provide applications, libraries, and frameworks explicit control over camera runtimes, through a precisely defined interface that enables cross-vendor portability of application code and easier system integration of new cameras and sensors. This API will also preserve application code across multiple generations of cameras and sensors and enable sophisticated control over the generation of sensor streams to increase the effectiveness of downstream processing.
The scope of the Khronos Camera API currently in design
Get Involved in Building the Open Standards You Need
The integration of increasing use of compute, vision and inferencing acceleration in embedded products is resulting in significant new business opportunities and Khronos is developing a growing family of open, royalty-free API standards relevant to meet the rapidly evolving needs of embedded and safety-critical markets.
Khronos remains committed to playing a vital role by providing a safe space for companies to cooperate to create open standards that benefit their own business and the wider industry. If your own company would like a voice and vote in any of these standardization activities, or you wish to implement a Khronos standard on your silicon or use a Khronos standard in your system design, Khronos warmly welcomes any company that wishes to participate.
