Granite River Labs, GRL
Kenichi Suganami
Mixed reality headsets have been touted by many to be revolutionary technology. Some users who have donned on the mixed reality headgear have been brought to tears, while others have been inspired to make tongue-in-cheek comments about screens being brought too close to our faces.
But what exactly is mixed reality, and what disruptive potential does it bring to have companies pouring in billions of dollars into its development? Can their present limitations be overcome, and will they truly transform our lives in the same way that portable media players, smartphones, tablets, and smartwatches have in the past?
As their names suggest, mixed reality headsets are worn on the head. Instead of a traditional screen, outputs are projected directly into a user’s eyes through high-resolution displays that envelop their entire vision. These displays can in turn be manipulated by eye tracking and hand gestures that replace the functions of keyboards, mice, and touch screens.
The latest mixed reality models have higher camera system performance that provide greater dynamic range and image resolution. Superior positional head and eye movement tracking also means that displays move in tandem with a person’s natural movements without disorienting the user.
Mixed reality is made possible through a process known as spatial computing. Just as humans make sense of the world around us through neural networks that tell us when an object is too close or too far, spatial computing does the same with sensors, computers, and actuators.
The first step in spatial computing involves lidar or radars that scan the surrounding environment using lasers or radio signals that bounce off surrounding objects. A process known as photogrammetry then makes sense of this information by combining information from multiple senses to produce 3D models. Recent developments in AI have helped systems produce even richer models thanks to neural radiance fields that fill in missing gaps in information that might not have otherwise been captured.
Machine vision then analyzes this compiled data to identify individual objects and how they might differ in design or behavior from other similar specimens. This allows mixed reality systems to flag out anomalies, such as defective products, appliances that have not been switched off, or poorly fitted installations.
When enabled, mixed reality systems can also take action on behalf of the user. For example, virtual displays might dim automatically when exposed to high levels of light. For higher level tasks, such as fixing broken plumbing, alerts can be sent out to human technicians for them to be deployed on site in real time.
The main difference between mixed reality and its more widely known cousins augmented reality (AR) and virtual reality (VR) lies in what is perceived and what users are allowed to manipulate.
In terms of what wearers will see, mixed reality headsets provide a digital overlay while still keeping the physical world visible. This means that if you’re opening up a home page, you’ll see the usual app icons floating above your room — or wherever else you happen to be using it. In this sense, mixed reality is not too different from AR, which aims to enhance real world interactions with statistics and other virtual details that tell users more about what’s in front of them. VR, on the other hand, is 100% synthetic and closed off from the real world.
Mixed reality headsets now come with a transitional knob that allows users to adjust how much they are exposed to the real world, making it possible for users to enjoy that truly immersive experience if they desire.
Mixed reality headsets allow life-sized models to be manipulated within real-world contexts
What truly makes mixed reality technology exciting is the doors that it opens up for real world manipulation. Mixed reality interfaces allow users to build and interact with life-sized models and witness first hand how these models can be positioned in physical spaces. The ability of latest models to capture hand and eye movements effectively also means that physical objects can be copied into virtual displays, allowing for possible interactions to be witnessed first hand. In fact, eye tracking technology has become so advanced that UI systems are now capable of highlighting things as you gaze upon them in real time.
According to Persistence Market Research, the global mixed reality market generated US$1.2 billion in revenue in 2022 alone. Projections predict that demand for mixed reality technology will continue to increase at 35.6% CAGR over the next 10 years and reach a height of US$24.6 billion by 2032.
Positive market outlook aside, it’s important to acknowledge the current investment landscape where both companies and consumers are wary of jumping the gun after seeing how blockchain technologies fizzled out just as quickly as they arrived. The good news is that there are plenty of reasons to be optimistic about the staying power and growth potential of mixed reality headsets.
For one, mixed reality tracking technology has reached a watershed threshold where it is now capable of real-time graphic display, granting users truly immersive experiences that do not require them to suspend their belief. Image streaming capabilities have also been reported to clock speeds within 12 milliseconds, eight times faster than the blink of an eye.
This opens up a whole new realm of user-cases for mixed reality headsets to be applied to, even if it doesn’t achieve mainstream adoption. Already, mixed reality headsets have been deployed to achieve greater precision on brain and intestinal surgery, and have also been adopted by militaries to enhance their situational perception.
Excel sheets and word documents probably aren’t the most exciting thing to view on mixed reality headsets, but with increased interactivity and greater potential for collaboration, these functions could be the very thing that changes the way that we work and play forever.
The virtue of mixed reality headsets being virtual means that the barrier to entry for high-stakes jobs can be lowered significantly, as trainees in construction, healthcare, and other high risk industries can undergo training at the fraction of original material costs, while also keeping both trainees and instructors safe. If those applications sound exciting to you, read on to find out more about this potentially groundbreaking potential.
Construction and engineering are notoriously stressful professions given the high material costs and how difficult or even impossible it can be to undo errors. Mixed reality devices allow engineers to practice assembly and experiment with designs without spending a single cent on materials. Furthermore, it opens up the possibilities for engineers to take work home and practice their craft from the comfort of their homes, potentially spending up their training trajectory and overall productivity.
Similarly, educational institutions often find themselves limited to what they are allowed to expose students to for safety and compliance reasons. Mixed reality headsets can allow students to gain much more from their learning experience by allowing them to interact with 3D simulations of chemicals, tools, and natural environments that would otherwise be dangerous. It also opens up the possibility of providing even more tailored learning experiences to cater to students with unique learning styles that would otherwise not be possible within a traditional classroom.
Perhaps the most obvious application of mixed reality headsets, this new technology will allow gamers to live out their wildest dreams by giving them a fully immersive experience of worlds that they have always dreamed of. This has the potential to not only enhance the experience of gaming enthusiasts, but also invite casual players to relive childhood experiences from a totally new perspective.
Mixed reality has already been deployed and proven in healthcare settings. By providing healthcare practitioners with real-time information about patient vitals and medication, this technology can significantly cut down time spent referencing charts and potentially eliminate fatal errors on the job. With the right enhancements, mixed reality headsets may also compliment commonly used medical tools by allowing scans and analysis to be conducted more efficiently.
Collaboration with global teams can become even more efficient by enabling life-like office tours and face-to-face collaborations without the hassle or cost of traveling across the ocean. If combined with translation applications, mixed reality headsets can potentially help global teams overcome language and cultural barriers as well.
Taylor Swift concert tickets may be limited, but when you’re performing on a digital stage, there’s no cap to the number of front row seats for global audiences to enjoy. Mixed reality headsets can allow users to get up close and personal with their favorite entertainers and sports icons, and even provide music and sports enthusiasts with breakdowns and analysis of expert techniques.
Adventures into little traveled parts of the world can be made that much more accessible to a wider audience with mixed reality. For example, travelers with limited limb mobility or who suffer from panic attacks can be transported directly to the depths of the ocean where divers feed live sharks. For tourism boards around the world, mixed reality can allow them to provide users with a much more realistic taste of what to expect and entice visitors from around the globe to pay them a visit.
It's no secret that the metaverse has struggled to live up to its initial promise. With a clunky user interface and limited appeal beyond gaming communities, many consider the metaverse to be yet another failed technological venture. However, that climate of opinion could be shifting with mixed reality headsets. With real-time motion tracking capabilities, these headsets have already replaced awkwardly rendered avatars with hyper-realistic digital personas based on scans of the user’s actual face.
Moreover, the involvement of major electronics brands, which boast loyal customer bases, in developing their own versions of mixed reality headsets will significantly bolster the metaverse user community. These factors could provide just enough push to tip the metaverse pass tipping point, unlocking an entirely new dimension of personalized commerce and leisure. Users would be able to interact and communicate with others across the globe in lifelike ways, all from the comfort of their own homes. This would also open up opportunities for innovative sales techniques, such as digitally created life-sized posters, demonstrations, and even full-scale virtual events.
To turn this vision of a hyper-realistic digital space into reality, manufacturers of mixed reality headsets must collaborate closely with connectivity and network providers. Seamless interoperability is crucial to enable the unrestricted flow of data within metaverse servers, ushering in unprecedented levels of connectivity and interaction.
With so much revolutionary potential on the table, you’ll definitely want to start pushing out your reality-enhancing products to customers before your competitors do. After all, mixed reality headsets aren’t cheap, meaning that customers will be much less likely to reinvest in your product once you’ve missed the first adoption train.
So if you want to put your AR, VR, or MR related tools on the fast track to production, reach out to GRL today and receive expedited and accurate testing services from leading engineers.
General Manager of GRL Japan, Kenichi Suganami
Ken has spent over 11 years at GRL Japan as the Regional Segment Marketing Director covering Automotives, Embedded, Smartphone, Gaming, and Digital Consumer product segments. Working deeply with major Japanese OEMs, particularly in digital consumers, game consoles, and LSI companies, Ken has comprehensive knowledge in Machine Learning, GPU, and CPU-based system architecture for digital consumer products.