As immersive technology is advancing how we can interact with digital immersive spaces and objects is changing so another important term to learn is Mixed Reality or MR. It is easiest to think of MR as AR but with advanced digital-physical interaction that narrows the gap between the real and the virtual. Although Mixed Reality can be accessed via a screen like AR, in the future MR will primarily be accessed via a dedicated headset leaving your hands free. For example, in a Mixed Reality experience a digital object such as a book can be anchored to a specific location in the real world, maybe your bedside table, and when you want to read that book you can put on your headset, pick the book up with your real hand and turn the pages with your real fingers as you read. When you are done reading you can put the book back on the bedside table and it will remain there until you need it again. There will be no touch screen involved, the technology allows the headset to track your hands and the furniture around you so you can manipulate the digital object and place it within the real world.
Some key features of MR:
The easiest way to understand what Mixed Reality is and how you can use it is to see it in action. Check out these top MR examples.
Another technology that is often confused with Mixed Reality and Augmented Reality is Virtual Reality or VR. While AR and VR do similar things the key difference is that VR fully immerses you inside a digital environment via a headset while an AR solution allows you to see the real world around you but with digital elements overlaid onto your environment. When trying VR experiences it is common for users to walk into tables and chairs or even other people because they can’t see the real world around them, only the digital world inside the experience, whereas AR users can see the world around them and navigate it as they would normally do.
AR stands for Augmented Reality. AR is a technology that overlays an interactive digital layer onto the world around us, it is not a fully immersive experience and the world around the digital object remains visible. All AR solutions do basically the same thing by superimposing 2D or 3D digital objects onto the world as you see it through a digital camera. These objects can be oriented and positioned over the real-world camera view and might be interactive, via your touch screen, or passive depending on the intended experience. The first widely available smartphone-based AR game that sparked the imagination of millions around the world was Pokemon GO from Nintendo and developer Niantic in 2016.
All these various technologies combined are referred to as XR. This is a broad term for all immersive technologies covering Mixed Reality (MR), Augmented Reality (AR) and Virtual Reality (VR).
Mixed Reality works via a digital camera and a screen or viewer, both mounted on a handheld device or Head Mounted Display (HMD), that the user looks through just as when you take a photo on a smartphone you are looking through the device's camera via the screen. MR software running on the device or in the cloud analyzes the camera input much like the user’s eye sees the world but before it displays a digital image on the screen the software uses advanced real-world spatial mapping via LiDAR, IR and GPS to scan the environment then map and track the user’s hands, eyes and body movement via IR, accelerometers and gyroscopes. The software then merges the real world with the digital in a way that the user and their real environment feel solidly connected to the digital one and the user is able to interact, touch and manipulate the digital world in a similar way to how they can the real world. Some Mixed Reality solutions also enable multiple users to interact with and manipulate objects in the same MR space.
The unique power of Mixed Reality over traditional screen or print-based media, be it gaming, advertising, marketing, film, TV or sports, is that immersive technology is measurably more engaging on multiple levels and pulls the user to directly interact with the experience and get involved on a physical and emotional level rather than passively observing it. This enhanced engagement can invest a user, player or customer in the game, product & brand ecosystem or shopping experience in ways still being explored. More than one user can also interact with a single shared digital object which opens up all sorts of future possibilities for training, gaming, and the development of new MR use cases.
MR is now available to everyone via affordable headsets such as the Zapbox, or the more expensive Meta Quest Pro, Varjo XR-3 or Microsoft Hololens the use cases for Mixed Reality are exploding. Industries such as engineering, aviation, and medicine are also embracing the ability of Mixed Reality to improve training outcomes, enhance collaboration and reduce costs in research, manufacturing and service sectors. As MR solutions become more affordable social media, entertainment and retail companies will become more engaged with the technology now they are familiar with the power of AR to engage an audience however, MR is yet to gain much traction in these public-facing sectors.
Industries including defense, aviation, engineering, robotics and medicine have invested time and money into exploiting MR for better training outcomes and to make huge efficiencies in manufacturing and development processes as well as health and safety. MR is also popping up in gaming for headsets like the new Meta Quest Pro but it is still early days.
As the popularity of Mixed Reality grows across multiple sectors and industries, encouraging research figures on its capability to engage, entertain and raise dwell times as well as boost memory encoding and sales conversion rates for brands, are further promoting AR and MR solutions amongst retailers, social platforms and entertainment companies. Below are a few key stats covering all XR solutions.
There are many tools out there for Mixed Reality creation but most require an advanced skill set and experience. Our affordable mixed reality headset, Zapbox comes complete with a Unity SDK to enable designers and developers to build feature-rich MR projects. There are also industry-standard tools available from Unity, Unreal Engine and Autodesk Maya.
If you’re on a budget and want to explore the possibilities of mixed reality without the hefty price tag our very own Zapbox provides a lightweight way to create and experience mixed reality with Unity. Zapbox comes with two active 6Dof controllers with buttons mirroring that of other popular headsets to maximize content compatibility.