Guidelines and best practice for shooting professional-quality video of your AR experiences. 

 

When filming AR content as it appears on a device, such as those featured on Zappar’s YouTube, Vimeo and social channels, there is a lot to consider - focus, exposure, lighting and location all play an integral part in showcasing the true value of your AR experiences.

For those of you in the Zapworks community who are looking for advice on how to make your AR experiences shine on camera, this post is for you.

I’ll unpack some of the lessons we’ve learned from shooting thousands of AR experiences, as well as the challenges faced and how to overcome them so you can film professional-quality video of your AR experiences every time.

 

---

 

Shooting in the Zappar app

Unlike most mobile apps, Zappar is showcased best when shot fully live. Usually, when shooting live footage of an app, the screen will be replaced in post with animated content, allowing most of the obstacles below to be overcome. However, as Zappar allows digital interactions with the physical world, recording the experiences as they happen is essential.

This presents a unique set of challenges. Device screens emit light and are highly reflective, which means that exposure, focus and surrounding light sources need to be carefully controlled to make sure the device’s screen is captured as clearly as possible.

 

An example of shooting content in the Zappar app, capturing multiple on-screen interactions.

 

Equipment

Shooting AR doesn’t require special camera equipment; most modern cameras and even smartphone cameras will shoot great quality footage. At Zappar HQ we currently use a Panasonic GH4, though we’ve also shot with Canon DSLRs in the past.

 

Exposure and focus

In order to try to expose for both the device’s screen and the surrounding environment, it is often necessary to adjust the brightness of the device’s screen, but be sure that the refresh rate of the screen does not cause flickering. We find that Android devices are far more likely to exhibit this issue than iOS devices. If you do experience screen flicker, try adjusting your camera’s shutter speed, as well as the device’s screen brightness, to find the perfect balance.

Likewise, maintaining critical focus on as much of the device’s screen as possible is the key here. Manual focus is recommended as the content’s motion across the device’s screen can easily confuse autofocus systems, resulting in a loss of focus.

 

Composition

A POV or over-shoulder style works well for showcasing mobile AR content. You can find loads of great examples on the Zappar website and social channels. If the experience is tracked to a target, a composition which shows both the target and the device screen (with the content displayed) is most effective in conveying the nature of AR. The hand position of the user will also need to be considered so that their hands do not get in the way of the screen and obscure content.

 

This Nespresso project was shot in a modern kitchen. The right of the frame shows the physical target, whilst the left shows the device and experience. This composition allows the relationship between digital and physical to be instantly conveyed.

 

Shooting for headset-based experiences

When recording content that is designed to be experienced through a headset, on-device screen recording is the most effective method. If possible, switch off the stereo option, unless your finished video is designed to be viewed on a device through a headset. To allow for smoother recordings, a stabilizing gimbal is a good option. At Zappar, we use the DJI Osmo Mobile.

 

An example of shooting headset-based experiences in our sports data visualisation project in Mixed Reality with Zapbox and Kindred Futures.

 

Location and lighting

Shooting inside is recommended, as this will give you more control over the environmental light.

If shooting outside is required, consider the position of the sun, cameras really struggle to pick up the light emitted from a device's screen when outdoors. This is due to how reflective device screens are, as well as dynamic range limitations; the camera’s ability to simultaneously expose correctly for both the device’s screen and the surrounding environment.

To help with this, utilise an umbrella or parasol to help shield the device’s screen from sunlight. A high-quality anti-glare film on the device’s screen can also help, although it can make the content appear washed out or fuzzy. Use of polarising filters on the camera lens can also help to cut down on screen glare.

It’s also important to consider an appropriate location in terms of setting the scene for your project. The Nespresso Inissia campaign video above was shot in a modern home kitchen, which is appropriate as the purpose of the campaign was to allow users to visualise the coffee machine in their own home. Shooting the experience in a different setting would remove some of the context of the campaign, reducing the video’s effectiveness in expressing the project’s message.

 

An example of shooting outside, with anti-glare screen film.

 

Audio

Making sure the device is not on silent, and has its volume up is of course a must (not to mention switching your camera’s mic on!) In addition, it’s a great idea to enable the capture of device audio when screen recording. This way, you can sync the clean audio from the screen recording in post, using your in-camera audio recorded live through the device’s speakers as reference.

This method is also handy as it allows for more flexibility in terms of filming location. You won’t be as reliant on controlling environmental sound, as you’ll be replacing it with the screen recorded audio. It’s a great idea to record 30-60 sec of atmos audio at your location, so you can layer this over the experience audio for a higher quality end video.

 

Device screen recording

Mentioned above as a benefit for recording clean in-experience audio, screen recording on-device is a no-brainer - it allows you the option in post to create in-sync picture-in-picture compositions, showcasing both the visual quality and interactivity of your experiences. Sometimes a 3D model just looks more impressive when you can see all the detail, and screen recordings show exactly what the user sees when they engage with your experience.

 

An example of a video featuring picture-in-picture with screencap. Along with the transition effects, this creates a visually rich piece, showcasing the quality of the 3D content, whilst retaining the interaction.

 

Final thoughts

Video remains the most effective and established medium to truly showcase AR. Particularly from a business development and marketing point of view.

Describing AR over the phone or in an email pales in comparison to being able to show the inherently visual nature of AR, as well as the context of the experience, it’s relationship to the physical world, and the sense of interaction. Video is the next best thing to experiencing AR in real life, so don’t underestimate its value when showcasing your projects.

If you get the majority of the ingredients right mentioned in this post, you should be well on your way to filming professional-quality video of your AR experiences that help you market and sell your AR services. 

 

---

 

About the author

Matt has been Zappar’s in-house videographer and photographer since 2015 and has extensive experience in shooting and editing all of Zappar’s augmented, virtual and mixed reality videos.