This year was my first time attending the Augmented World Expo – since I'm a composer working mainly in opera, it isn’t often that I get asked to speak on a panel at a technology conference. But I was delighted to be there because my own work has often focused on new technologies (including my VR opera The Parksville Murders, my data-mining show Looking at You, and my upcoming AR opera Joan of the City).

I’ve become increasingly convinced that there is a place for emerging tech in opera and in the performing arts in general. People are often initially skeptical when I talk to them about incorporating XR into opera. As a pretty old art form with a reputation for stuffiness, opera doesn’t immediately seem like a natural fit for XR experiences. But I’d like to argue that, rather than being odd bedfellows, XR and the performing arts (including opera) are made for each other.

Historically, theater and opera have long been early adopters of technology, pioneering technical feats in stagecraft including lighting, moving sets, and even holographic effects like the Pepper’s Ghost. Indeed, the orchestra itself can be considered a technological innovation, in that it allowed acoustic sound to reach decibel levels unheard of (pun intended) before the invention of amplified sound. Both opera and theater are art forms that innovate in the service of storytelling, much like XR experiences are currently doing today. 

  

“The elements of stagecraft developed for live theatre are particularly applicable to building XR experiences. Because the technique of stagecraft was developed in the time before filmmaking, it deals with the entire three-dimensional playing space. “

Whereas film technique is all about controlling the gaze of the viewer, in a live theater setting the audience can look wherever they want. This has meant developing other methods (including the aforementioned lighting and set design) for focusing an audience’s attention. Theatrical shows must be able to change an audience’s sense of time, space, and motion without being able to cut away as you can in a film. While XR experiences are mediated through cameras and screens, they are more akin to live performing arts than to film in that the audience can similarly choose where they want to look. Because of this, XR experiences must also be designed for the entire three-dimensional environment. Therefore, working with experts in theatrical stagecraft can serve to create even better experiences in XR.

The performing arts also have a lot to gain from working with XR developers. At the most basic level, incorporating AR elements into a show is one way of heightening the stage picture. But beyond this, it can also help to bring the show outside the theatre. In addition to creating greater accessibility for those who may not otherwise be able to attend theater or opera, in our current moment, untethering from the physical theater space could actually help to bolster the performing arts. As of this writing, most arts venues in the US have shuttered. While some organizations in the EU have begun programming concerts again, they must generally still operate at partial capacity.  Especially in the US, arts organizations are looking for new ways to bring live performance back to their audiences. XR environments, particularly VR, could provide a solution to this problem. Further, the sound spatialization possible in XR environments is a better representation of the live concert or theatre setting than 2D sound can ever be. 

  

But this brings me to another point. Sound has often been among the last considerations for XR developers. We so often prioritize the visual elements of an experience that we don’t think about how sound can also be used in worldbuilding. Because of this, the sound design in XR isn’t always as well thought out as the visual elements. And this is a sticking point for most people working in the performing arts, especially opera. To truly realize the potential of collaboration between XR and the performing arts, we will need to ensure that there is more thought given to sound quality and latency issues. This is especially important for live streaming performances where sound quality is often sacrificed to ensure the stability of the video.

In conclusion, there is a great deal of opportunity in seeking collaborations between the performing arts world and the world of XR. As we find our way in the post-COVID environment, pursuing this opportunity will not only serve to strengthen both fields, but it could lead to new and exciting forms of artmaking that we can’t even predict.

 

 

 

 

 

 

 

 

  Kamala Sankaram, Composer