While there are a number of different types of AR, perhaps the most engaging and exciting form uses fancy computer algorithms to analyse camera images looking for pictures to bring to life. This is called vision-based augmented reality and it's at the heart of most modern AR apps, such as Zappar. These apps are super-carefully crafted so they can analyse at least 30 images every second, giving the user a fluid and rich 3D experience. The best apps feature diverse interactive platforms allowing for immersive game-play, the ability to link through to other digital experiences online, and the sharing of photos or videos.
The term Augmented Reality just doesn't roll off the tongue so we came up with our own lexicon with Zappar. The clue is in the name: Zappar is a free-to-download APP on iOS and Android, using AR that allows you to ZAP things in the real world and bring them to life.
It's our mission at Zappar to make the world more fun one zap at a time. If your TV remote control was your Zappar in the 80’s to change channels then your smartphone, with the Zappar app installed, is the new way to zap the world around you making all objects and images potential multi-media channels serving additional, engaging, value added content.
While Augmented Reality may seem ultra-modern, the principles behind it have been around for some time. As far back as the 1950s and 60s clever folk in leading Universities were conducting extensive experiments on "human-computer interaction". It was an exciting time to be a nerd – "virtual reality", "portable computing", "immersion", all previously in the sphere of science fiction, were now being crafted and defined in labs around the world.
Indeed Zappar's own algorithm and platform has its genesis at the University of Cambridge. Two alumni of the Engineering Department there - Dr Simon Taylor and Connell Gauld – built Zappar's tech from the ground up. Simon's PhD is titled Fast Object Localisation for Mobile Augmented Reality Applications. We've got a copy in the office but it's far too complicated for the rest of us to understand. Connell built the platform (around Simon's algorithm) that can describe and display immersive 3D experiences. They're now joined by a fantastic team of technical and creative types who specialise in building and delivering the characteristic Zappar magic wherever it goes.
In the past the sheer computing power needed to augment reality wasn't very accessible – it was pretty expensive! While this made it difficult for consumers to adopt AR, it did see a good number of business and military uses. From automotive design to aircraft assembly, architecture to archaeology, fighter-jets to medicine, it's made it easier and more efficient for people to do their jobs. Do you remember Maverick's fighter-jet heads-up-display in the film Top Gun? That's a form of AR. You may even have it in your fancy new car – not in the same way as Maverick had in his jet of course but to show navigation cues or speed information instead.
As more powerful personal computers with webcams emerged, consumer friendly AR became more of a reality. It began to be used by brands and their agencies in marketing campaigns, with early adopters including the likes of Coca-Cola and Marvel. To run these PC-based experiences users had to download the AR application to their desktop. Marvel's AR application for the Iron Man movie allowed users to wear a 3D Iron Man mask when sat in front of their computer’s webcam and monitor. While there was a degree of novelty about these experiences, being tethered to a laptop or large screen meant the potential consumer application was limited.
AR also entered the mass market in the late 90s and early 2000’s through gaming. The likes of Nintendo Wii, Sony PlayStation and Microsoft Xbox allowing the detection of player’s movement to experience digital game play in a real world environment.
With the dawn of smartphones, with better cameras, faster processors, greater network coverage and download speeds; AR made the leap to mobile. This was a ‘game changer’ for the technology as smartphone prices tumbled, penetration grew exponentially worldwide and the app store reinvented whole industries. At the same time the slow rise, acceptance and mass awareness of QR codes taught the world to scan and helped people make the connection between a device and the physical world around them via an app. AR is the natural progression of this scanning function on mobile but this time delivering engaging and valued experiences as infotainment rather than the static link to a URL that you get with QR codes. Augmented Reality on mobile begins a new era of connectivity between the physical world and digital devices, opening up a hidden world of additional content in the things we see around us. It takes the experience of looking down at your phone, disconnected from the everyday, and makes you look up and see the world with a different lens.
As a user the process is simple – you download the free Zappar app which opens up a camera view in "scanning" mode. Once the app detects something that's Zappar Powered it assembles all the content and brings that thing to life on-screen in front of your very eyes.
As for the phone, there's a lot going on behind the scenes! We'll save you (and us) from the 100+ pages of technical documentation underlying the algorithm but here's a potted outline of what's happening.
30 times a second the app reads an image from the camera and searches it for little distinctive corners and shapes. Once it's found a good number of them it compares them with a database of corners and shapes that it knows to be looking for – the distinctive features of the "target" image that it's going to bring to life. Matching features let the app know not only that it's found the target, but also where that target is in the camera image. Armed with this knowledge the app places the 3D experience in that place. Huzza! The user is amazed, and may have let out a small scream.
The problem a lot of other AR systems face is the issue of having to detect thousands of different target images at the same time. It's easy to understand why this is hard when you consider that a lot of images look very similar – say photos of babies, or different crops, scales and localisations of the same marketing print campaign. This makes it difficult for these apps to tell the different images apart. Other AR providers have to invest in costly infrastructure to try to deal with this, and some have to hoist up their prices in order to keep demand low and even then only allow images in their systems for short amounts of time.
Finally, there’s the related issue of ‘image squatting’ in that you can only serve one piece of AR content off a single target image. What if person B wants to use the very same image as person A for their AR experience?
Our patented zapcode solution overcomes all these issues in one go. A zapcode looks like this:
The zapcode is super versatile. It's a little icon that lets the user know there's AR content available. Otherwise how would they know it's there!? You'll notice that other AR providers add an icon to images too for this very reason.
But the humble zapcode does so much more.
Surrounding the zapcode is a special arrangement of bars called "bits". These tell the app which piece of AR content to download and augment on the image. It doesn't matter if the image looks similar to others in the system – since it's the code (rather than the image) that identifies the content, you're always going to get the right experience.
The clever clogs at Zappar have designed the zapcode so that there can be 4 billion different unique codes. That means we're not going to run out any time soon and that all content can continue being served forever – there are no time limits and no extortionate monthly fees needed to keep your zaps working in Zappar. The industry lingo for this is scalability of search space, and zapcodes have it in spades.
Since user experience is so important, we've worked hard to ensure that zapcodes are scanned really quickly by the app even when they're tiny in the camera view. It only has to be 8mm in diameter (less than half the recommended size of a QR code) to work well and you can track your AR experience off the code itself rather than a whole image if you like. Think of it as a QR code on steroids and the evolution of brand devices as a gateway to multi-media content.
That’s why we’ve introduced ZapWorks: an online tool that allows users to create and design their own AR experiences either off a zapcode itself or a target image featuring a zapcode. It’s super simple to use and caters for people of all technical ability.
It’s this democratisation of augmented reality that will determine its future value as a technology and platform for true mass adoption. We’re looking to put the creative tools in the hands of the people to see how they use and adopt it to deliver bite-size infotainment experiences in the world around them, connecting digital devices to physical things.
As we witness further advances in both software and hardware along with increased network speed and coverage, and the advent of new wearable products from Google Glass to Oculus Rift through virtual reality, AR is increasingly becoming a fascinating interface and facilitator for deeper, more connected social experiences.
No one can predict the future but the reality is that AR is consumer ready right NOW. The technology works and has been proven over many years and many platforms. It’s really the experiences that AR delivers that will show that AR is neither a gimmick nor a technology looking for a solution. As an industry we need to put people at the heart of the experience and determine how AR makes their lives simpler or better versus other solutions rather than working from the technology back to the individual.