Last year, I shared my views on some of the challenges the mobile VR industry was facing. I felt that while the hype was high, delivery on what was promised was nowhere near expectations. Many of the systems I saw were still tethered to a PC or needed a heavy battery in a head-mounted display (HMD). They were expensive, dependent on external sensors and, at that stage, were supported by limited quality content. All of this made them fun devices for early enthusiasts but unlikely candidates for mass adoption.

How has the picture changed after one year? We saw progress, but more incremental than ground-breaking. There are still significant barriers in true hardware mobility, user experience and cost.

The AR/VR/MR Market Rebound

Growth is now expected to return to about 50 percent year-over-year (units shipped) after an air-pocket in 2017, especially thanks to Google and Facebook upping the user-friendliness game. IDC estimates that standalone and tethered headsets will quickly bypass screenless headsets, reaching 85 percent of shipments by 2022. A Canalys report projects standalone headset volumes to grow at a CAGR of 140 percent from now until 2021 (Figure 1). Content and delivery are improving, at least in the VR space, and new VR devices (Facebook Oculus Go, HTC Vive Pro and Lenovo Mirage Solo with Daydream) are expected to ship soon. And the much-hyped Magic Leap One MR headset finally emerged.

Despite this growth, there’s still much room for improvement.

Great potential, still not practical

At the recent Mobile World Congress (MWC) in Shanghai a number of AR/VR products were still tethered, perhaps avoiding the impracticality of “mobile” solutions that simply put a support PC in a backpack and tie it to several wearable sensors. However, these looked awkward and limiting to a total immersion experience.

Most HMDs still depend on external tracking sensors positioned around the game area – one sensor (of four) is circled in Figure 2. These external sensors also feel impractical for consumer setups; you have to reserve a large game space between the sensors and run power cords (potential trip hazards) to each.

My Experience? Still a Lack of Quality Interactive Content

In an improvement of sorts, HTC showed off the Vive focus with the Karting Car mockup. This headset is actually standalone and is definitely an ergonomic improvement over most mobile options at MWC Shanghai. It supports inside-out six-degrees-of-freedom (6DoF), but likely has limited horsepower compared to tethered HMDs. So, I checked it out.  

Sitting in the Karting Car, I adjusted the HMD and a virtual reality scene was transmitted wirelessly (Figure 3). I could move my head and look around, which was nice. Now my expectations were high. I thought, “OK what’s next? Can I interact with the Karting Car?” Apparently not – that was the complete demo! It was a nice upgrade over Google Cardboard but nothing more.

My hunt for better interactive content continued by testing a decent fitness VR game. In the game, you ride a stationary cycle but see yourself chasing a pack of horses; the goal is to lasso one (Figure 4). And of course, the slower you pedal, the faster the horses run away from you.

Considering it’s a tethered game (with a customized setup to allow user interaction), the player can still get a great 25-minute workout and have fun doing it.  

Overall, we saw some AR/VR/MR progress at MWC Shanghai this year, but nothing to get excited about. There are still significant barriers in what most consumers would consider mobility, user experience and cost. The systems we saw were concepts, not ready for prime-time and still failing to show progress in interactivity. They are nowhere near ready for mass consumption, but still fun for conference attendees.

We Have to Change the Game

CEVA believes that an ideal VR/AR/MR experience should be based on a standalone HMD with sensing and tracking driven from the HMD (inside-out rather than outside-in), perhaps with a pointer/control device and, if needed, a portable power pack. This is what a reasonable user would expect. Magic Leap seems to have followed this philosophy; however, with their One MR headset release being so recent, we’ll have to see how it’s received.

The Magic Leap One uses the NVIDIA Tegra X2 based on a quad Cortex A57, a dual NVIDIA Denver 2 and an NVIDIA Pascal-based GPU with 256 CUDA cores. That’s a lot of horsepower, designed primarily (per NVIDIA) for self-driving cars and video cockpits with less demanding power constraints.

The biggest constraint to mobility in a standalone HMD is the battery, even if attached to your belt. The biggest constraint on the battery is power consumption for graphics and vision capabilities (positioning, tracking, etc.) and reasonable playing time (i.e., a few hours).

So how long will the Magic Leap One battery last? That information is still not known. We’ve heard that the system gets two of the A57's four cores and one of the two Denver 2 cores. Apparently, game/app developers must shoulder responsibility for a reasonable balance between function, performance and power on the rest of the cores.

We have to showcase the practical possibilities of VR/AR/MR by requiring a change in the approach to hardware design. For example, game and application developers should be able to offload vision-intensive tasks from CPUs and GPUs to a much more power-efficient vision processor. Tasks that naturally fall into this class will become increasingly important in advanced VR/AR/MR and include distortion correction, foveated rendering, inside-out tracking, sensor fusion, AI and potentially much more.

CEVA is taking capabilities like this to responsive design.

eletter-09-4-2018

eletter-09-5-2018