Beyond the Canvas: Building a Living Exhibition with WebAR

What happens when painting doesn’t stop at the canvas?

Carne da Terra (Flesh of the Earth) explores the relationship between the human body and the natural world. It blends painting, installation, and augmented reality into one connected experience.

Created by artist Maria Antonia, with AR development by Fábio Fonseca, the project treats digital space as part of the artwork, not an add-on.

Instead of looking at a painting, visitors step into it.

Why AR? 

The primary objective of the project was not to “digitise” the exhibition or increase novelty. It was to expand the expressive language of painting beyond its physical constraints and invite audiences into a more active, embodied relationship with the work.

Maria and Fábio chose AR precisely because it mirrors the project’s conceptual foundation. The digital layer remains invisible until activated, reinforcing the idea that meaning is not static but revealed through interaction.

In practical terms, this translated into a set of clear goals:

• Encourage visitors to spend more time with each piece
• Shift the audience from passive viewing to active exploration
• Bridge traditional fine art with contemporary digital practice
• Situate the exhibition within a broader conversation about technology, ecology, and the future of artistic expression

For creators, this is a critical distinction. AR is most effective when it is aligned with the intent of the work. When it is not, it risks becoming ornamental.

The experience

The experience was delivered through WebAR, allowing visitors to access the digital layer directly through their smartphones without downloading an app. This decision reduced friction and ensured that the technology did not become a barrier to engagement. 

Once activated, each painting reveals an extended digital layer. Elements begin to grow, shift, and transform, often echoing the organic forms already present in the artwork. Rather than introducing something entirely new, the AR builds on what is already there, giving the impression that the painting has simply continued beyond its physical limits.

Creative approach

• Each artwork was analysed compositionally to determine where and how digital elements could emerge
• Animated 3D assets were designed to reflect the same visual language as the paintings
• Spatial alignment was carefully refined so that digital content felt anchored and intentional
• Entry points, such as QR codes, were integrated subtly to avoid disrupting the exhibition environment

The overall effect is less about spectacle and more about continuity. The digital layer feels like a natural extension of the work, rather than an overlay sitting on top of it.

Designing a continuous spatial system

One of the most significant creative decisions was to move away from treating each artwork as an isolated AR activation. Instead, the team approached the exhibition as a unified spatial system.

This meant thinking beyond individual image targets and considering how visitors would move through the space as a whole. The AR layer needed to respond not just to specific artworks, but to the viewer’s position and trajectory within the gallery.

Challenges encountered

• Maintaining reliable tracking across varying lighting conditions and distances
• Designing a system that worked cohesively across multiple artworks
• Ensuring that interaction remained intuitive without requiring explicit instructions
• Preserving the tactile and visual richness of oil painting while introducing digital elements

Balancing these factors required careful iteration. The goal was always to support the artwork, not distract from it, which meant restraint was just as important as technical execution.

Technology and workflow

From a technical perspective, the project demonstrates why WebAR has become such an important medium for spatial experiences.

Using Mattercraft, the team was able to build and deploy the experience directly to the browser, removing the need for app downloads while maintaining a high level of visual fidelity.

Tools and process

• Image tracking to establish a direct relationship between the painting and the digital layer
• Mattercraft's Immersal VPS integration to create a digital twin of the exhibition space 
• 3D modelling and animation software for asset creation
• Asset optimisation in Mattercraft to ensure smooth performance across devices
• Iterative testing within the actual exhibition space to account for real-world conditions

Why WebAR matters

For creators, this is one of the most important takeaways. By keeping the experience browser-based, the barrier to entry is significantly reduced. Visitors can engage instantly, which in turn increases participation and reach.

It also simplifies distribution. A single link or QR code becomes the gateway to the entire experience. 

Results

While the project is rooted in artistic exploration, its impact can be understood through audience behaviour.

Visitors spent more time engaging with the works, often moving around the space to explore different perspectives on the digital layer. This shift from passive viewing to active exploration is one of the clearest indicators of success in AR experiences.

The integration of AR also introduced a new layer of accessibility for audiences already familiar with digital interaction, while prompting broader conversations about how traditional art forms can evolve alongside emerging technologies.

Key takeaways

For anyone looking to build similar experiences, there are a few clear lessons that stand out.

• Start with the idea, not the technology – the strength of this project comes from how closely the medium aligns with the message
• Design for space, not just screens – Consider how users move, where they stand, and how the environment shapes interaction
• Prioritise integration over impact – Subtle, well-integrated AR often creates a more lasting impression than highly visible effects
• Reduce friction wherever possible – WebAR provides a practical route to accessibility without compromising on capability
  
  At its best, AR doesn’t compete for attention. It deepens understanding.

Ready to start building?

If you’re exploring how to bring physical work into a spatial context, this is a great example of what’s possible with the tools available today.

Start by thinking about what your work could become if it were no longer confined to a surface. From there, consider how AR might extend that idea in a way that feels natural and meaningful.

And when you’re ready to build, start a free 14-day Zapworks trial.