Software-Defined Displays: The Secret Sauce for AR Smart Glasses
March 26, 2024
Sponsored Story
Smart glasses for augmented reality (AR) are almost ready for mass consumer adoption – but they’re not quite there yet. The major issue that has to be overcome is the display system, which is often too big and heavy.
For example, the Microsoft HoloLens 2 is a beautiful device, but it is simply too bulky for users to wear for extended periods of time.
To achieve a truly immersive, realistic experience with AR smart glasses, the display system must be very accurate, and deliver high resolution and low latency, whilst being bright enough to view in direct sunlight.
Typically, to meet such demanding requirements, the display hardware ends up being overly complex and requires extensive calibration – which translates directly into increased size, cost, and hassles.
So instead of a hardware-only design, TriLite has gone outside the box and shifted the complexity of its laser beam scanner (LBS) projector from the hardware to the software domain. By adopting a holistic ‘software-first’ approach from design through to manufacturing, TriLite’s Trixel® 3 LBS projector becomes ultra-compact and ideal for cost-effective mass manufacture.
This article explains how this shift to software has been achieved, what it means in practice, and how it embodies the ‘secret sauce’ that enables cost-effective, compact, and high-quality displays – clearing the way for AR smart glasses to break into the mainstream.
Software-First Approach
Trixel® 3 is a software-defined LBS display technology, with proprietary algorithms that ensure optimum image quality and minimal latency. The software controls the position and brightness with subpixel accuracy and achieves very low power consumption. This approach has enabled TriLite to build an ultra-small display system based on reduced complexity.
The three main pillars of LBS software are:
- LBS design
- LBS control methodology
- Manufacture and calibration
Figure 1: TriLite shifts complexity from hardware to software domain
Design: Tool Chain
Product development starts with careful, intelligent design.
At the design level, TriLite has developed a proprietary tool chain where it simulates and optimizes all parameters from ‘video-in’ to ‘photon-out’. This process includes everything from meticulously defining all the drivers that are used, even going all the way down to the level of the physics of the laser diodes themselves.
It is possible to precisely simulate what happens at the point of video input and where each photon lands on the display. This process includes ultra-precise pixel-painting simulations and 3D ray-tracing simulation algorithms.
The software simulations also include Lissajous scan patterns (as shown in Figure 1). These move control overhead into the digital domain, reduce MEMS mirror complexity and size, and minimize MEMS mirror power consumption. Lissajous patterns eliminate scanline artifacts at the borders and have a higher fill factor.
Device: trajectory control module (TCM)
The second level comprises the device, where proprietary algorithms for the device control are implemented. This enables the correction of any distortions and various beam offsets within the system. The key is that all these functions are performed using software in an extremely efficient and optimized manner to ensure optimal image quality.
TriLite’s Trajectory Control Module (TCM) controls position and brightness with subpixel accuracy. Furthermore, TriLite’s software adjusts the brightness of individual pixels on the fly to enhance image uniformity at the eye level, while simultaneously compensating for any non-uniformity of the optical combiner.
The TCM enables the computational resources of tiny state-of-the-art host SoC (System on Chip) processors to be leveraged as they continue to improve following Moore’s Law. This ensures exceptionally efficient performance, whilst helping to minimize the complexity and size of the optical system. The fully optimized algorithms of the TCM system reduce computation overhead to a minimum, which significantly reduces system latency.
Manufacturing: Calibration System
The third and final level of the ‘software-first’ approach is in the manufacturing process itself, where an end-to-end calibration system is implemented to leverage mass production by utilizing machine learning software.
Leveraging software flexibility rather than relying on rigid hardware design allows TriLite to loosen alignment tolerances during manufacturing, which has a huge impact on the final yield, and ultimately, on manufacturing costs.
Analysis has shown that the alignment tolerances in TriLite’s system can be loosened by up to a factor of three compared to standard positioning tolerances.
TriLite’s fully automated machine-learning calibration system enables high-volume mass production and a higher assembly yield. At the end of the production line, after a full set of AR glasses has been assembled, the calibration engine can efficiently calibrate for all manner of distortions in the system. This calibration process includes compensating for distortions or brightness non-uniformities introduced by waveguides or other types of optical combiners.
Figure 3: End-to-end calibration
Conclusion
TriLite has taken a unique ‘software-defined display’ approach, shifting complexity away from hardware, and instead has applied proprietary algorithms to deliver optimum image quality, reduced latency, and lower power consumption.
Creating the software and integrating it into hardware is not a trivial matter and requires a highly competent, interdisciplinary engineering team with a broad range of expertise ranging from optics, MEMS, electronics, and mechanics to software and algorithms.
Ultimately, TriLite’s ‘software-first approach’ to the field of AR glasses enables considerably faster development cycles and low-complexity manufacturing, where the proprietary software stack delivers the smallest size, the best image quality, and the lowest manufacturing costs possible. The result is that the consumer will finally have access to lightweight, discreet, and comfortable AR glasses at an affordable price, which will lead to a wider adoption of this powerful technology.
Dr. Manuel Dorfmeister is an expert in microelectromechanical systems, with a PhD in Piezoelectric MEMS development from the Technical University of Vienna, as well as numerous patents registered and papers published. In 2021, Manuel was appointed head of the MEMS & Software department at TriLite, the company that is designing and building the world’s smallest projection displays for Augmented Reality mass market devices.
