You may have heard of AI.
We certainly have here at Zappar. In the last year, we’ve seen an explosion in the effectiveness and utilisation of AI technologies across industries. Software developer tooling, in particular, has seen great strides recently with jaw-dropping AI models from the likes of Anthropic and OpenAI powering the next-gen coding experiences of Copilot, Cursor, Windsurf, Claude Code, and others.
Mattercraft is our complete 3D development environment for the web. It enables creative developers to build rich, interactive 3D experiences that are delivered to end users in the browser. It ‘bridges the gap’ between the fantastic visual tooling we’ve come to expect from game engines (like Unity and Unreal), and the more code-oriented world that’s been the mainstay of frontend web development. As a result, Mattercraft is great for creatives with or without coding experience.
This month, we’re releasing AI Assistant - your helping hand for building next-gen 3D experiences for the web. It’s fully integrated into Mattercraft and provides several tools that take the creative process to the next level. From helping you learn and understand Mattercraft, all the way to vibe-crafting (creating content and code entirely through conversation with AI), the AI Assistant supercharges the intersection between visual and code development, just as Mattercraft itself does.
In this article, I'd like to tell you about AI Assistant - how to use it and what it’s capable of - but also to give some of the background around its implementation and why Mattercraft is such a great foundation for this technology.
An important note before we get started. As artists and developers ourselves, we see both the incredible opportunity of AI and feel the creative and economic threats that it represents. Our AI ethos is that AI works best when it superpowers your creative process. It’s about helping you to create amazing content efficiently, with confidence, and about removing barriers to the realisation of your creative vision. We believe that AI Assistant achieves just that - it works for you, not the other way around.
AI Assistant has three modes, each fully integrated into Mattercraft:
Together, these modes can help new users get up to speed with Mattercraft and can dramatically improve efficiency in the creative process.
Let’s look at each in a bit more detail!
We’ve provided AI Assistant with a bank of Mattercraft knowledge that allows it to answer your questions reliably and comprehensively. Ask Mode sits as a panel on the right of the Mattercraft user interface and provides a familiar chat interface. You can freely discuss Mattercraft and your project in this view, and Ask will respond promptly and accurately.
In addition to knowledge, we’ve given Ask mode the ability to direct you to parts of the Mattercraft user interface. It’s therefore able to let you know about features you may not have encountered before, where contextually relevant, and help you to find them.
👉 Our AI ethos in action: We could have given Ask the ability to simply open areas of the UI itself. Instead, we show you where to go so you can find it easily next time yourself.
Let’s take a look at an example.
Given the prompt “How can I test my project on a phone?”, Ask mode directs the user to the Live Preview feature, describing how it can be used.
Ask mode is able to explore your project. It can analyse the contents of 3D scenes, animations, assets (including 3D models), available and installable add-ons, and any errors or issues that may be present - all so it can provide advice that’s directly relevant to your project.
In our second example, we ask, “What add-ons might help with my project?”. AI Assistant requests a list of recommended dependencies and describes relevant entries. It also helps the user find the Add-ons & Dependencies browser.
It’s in Edit mode that AI Assistant can really transform how you build and iterate on content. In addition to everything that Ask mode is capable of, Edit mode can propose actual changes to your project, including:
AI Assistant is able to propose these changes across your project, working with multiple files at a time. In each case, Mattercraft gives you the opportunity to accept or reject the changes, either individually or as a complete set.
👉 Our AI ethos in action: With the ability to accept or reject each change the AI proposes, you’re in complete control of your project.
To demonstrate the power of Edit mode, let’s take a look at it in action.
Our first example shows how AI Assistant’s Edit mode can propose changes to the contents and structure of your 3D scenes. It features an augmented reality ‘face filter’ experience.
Responding to the prompt “Make an animation that spins the hat and play it when the user taps on the hat”, AI Assistant takes the following steps:
PlayLayerClip behavior, correctly concluding that it can be used to play an animation in response to a user ‘click’ event.rotation property, correctly formatting the required values and setting an appropriate ‘easing’ function.PlayLayerClip behaviour attached to our hat object, and sets properties such that it plays the new animation when the onClick event is fire.As you can see, this example demonstrates that AI Assistant has an in-depth understanding of the concepts and structure of a 3D Mattercraft scene, and is able to propose useful and correct modifications in response to our requests.
Let’s take a look at another example.
AI Assistant has a comprehensive understanding of scripting and is able to write complete custom components and propose changes to existing script files.
In response to the prompt “Create a custom component that’s a dancefloor”, AI Assistant performs the following actions:
Mattercraft’s 3D editor (and Live Preview) are powered by its built-in bundler. This is a piece of technology that runs continuously in the background, compiling code and assembling assets for the purposes of running it safely in your browser or on a remote device (like your phone). As a result changes will appear, in real-time in the 3D editor, as the AI Assistant proposes them. You can literally watch it work.
A second prompt asks the assistant to “Now make the robot dance”. AI assistant takes the following steps:
You can continue to discuss the modifications that AI Assistant proposes in the chat - requesting additional changes, for instance.
Let’s take a look at a final example that demonstrates how well AI Assistant can work directly with Mattercraft’s user interface.
In this example, we ask AI Assistant to “Make a custom shader that’s a cyberpunk vibe for this drink's can”. For brevity, we’ve snipped out the steps where the AI is writing the shader (it’s the same process as the previous example).
As before, the result of the assistant’s work can be seen directly in the 3D editor. In this case, the AI has added configurable properties to the custom component that it generated. These can be edited directly in the Mattercraft user interface, without having to modify the underlying shader code. The assistant understands how the code it writes translates to reusable, configurable components:
As you can see, Edit mode is an incredibly powerful tool for working with 3D interactive experiences. Whether you’re a die-hard three.js coder or prefer to work visually, Edit mode can improve your efficiency and can help you build functionality that was out of reach or time-consuming to achieve.
If you do want to delve into script, then the Code Complete mode is here to accelerate your productivity. It works just like auto-complete in a script editor - providing code suggestions for your current cursor location that can be accepted with the ‘tab’ key.
The suggestions, however, are generated by AI Assistant, including the broad context of the file you’re working in. They can comprise of anything from the remainder of your current line, through to entire functions or code blocks.
👉 Our AI ethos in action: each potential completion shown can be accepted or rejected with the tap of a button.
This example shows Code Complete in action:
As you can see, it’s able to construct correct code from just comments and the contents of the file. This workflow - commenting as you go - gives Code Complete plenty of knowledge with which to suggest intelligent code, plus it makes it easier for future readers of your code (including Edit mode!) to understand your intention.
AI Assistant’s code knowledge is broad and spans multiple languages. This example shows editing a CSS file in real time. The suggested styling for the button (inferred by the assistant from the ID of the button alone) is a great start from which you can tweak to get the exact look you want.
Like most AI-powered tooling, AI Assistant leverages Large Language Models (LLMs) from some of the industry’s most innovative AI model providers, including Anthropic, OpenAI and Mistral.
Edit mode currently uses Claude 4 Sonnet - the brand new, powerful, while efficient, model from Anthropic.
At this time, Ask mode uses Claude Haiku 3.5 - it’s a smaller model that’s faster and more efficient than that used by Edit mode.
And finally, Code Complete uses Codestral - a model from Mistral that’s been specifically trained to be super fast and accurate at small-context code tasks.
As new models become available (from Anthropic and others) we’ll evaluate and update the default models used by each mode so they become better yet at helping you with your project.
Each Zapworks plan will come with an included allocation of AI credits. The credits can be used with whichever AI features you’re working with, meaning you won’t be wasting allowance on modes you don’t use.
Here’s how each of the modes stack up:
If you run out of included allocation, you can buy top-up bundles to get you back up and running with the AI features again.
We’re planning on introducing new AI modes and features in the near future, and the same credits will work there too.
A founding goal for Mattercraft was to allow our users to leverage the fantastic ecosystem that surrounds web development, allowing content developers to bring their existing knowledge, to benefit from the wealth of online support material, and to integrate Mattercraft into their existing infrastructure and workflows. While you absolutely do not have to be a frontend web developer to use Mattercraft, if you are, then you’ll find its concepts map directly onto your experience and understanding.
This foundation is not only great for helping new and existing users in making great content - it also means that the AI models (which include all that ecosystem knowledge in their training data) already know how to work with the underlying concepts and technologies in Mattercraft.
Mattercraft is built on a component model - each of the elements of your creative experience (the 3D objects, groups, AR anchors, cameras, etc) are described in a simple structure with inputs (properties) and an output. Those structures (components) are then assembled into the hierarchical tree that is your scene.
The component model has many benefits:
It’s this last point - abstraction - that’s the biggest benefit to AI Assistant. Mattercraft’s component model means that we can provide AI Assistant with a concise representation of the context of your project. That makes the AI faster and more likely to provide accurate responses and to propose correct changes.
It also means that, when AI Assistant is writing components of its own, it’s able to provide you with easy-to-use functionality that can be configured in Mattercraft’s user interface without having to touch the code.
While Mattercraft itself is generally available, we’re rolling AI Assistant out over the coming weeks in beta. During this time, we’d love to get your feedback on the assistant - we’re particularly interested in:
Beneath every AI Assistant conversation are a pair of 👍 👎 buttons. These allow you to share your chat with us (should you wish to) so we can improve the service.
If all this sounds like something you’d like to try, you can register to join the beta here.
👉 Our AI ethos in action: AI in Mattercraft is opt-in (during beta and beyond) and won’t be turned on unless you’d like to use it
We believe that AI Assistant can revolutionise the way interactive content for the web is built. These three powerful modes, Ask, Edit and Code Complete, work for you to supercharge your creative process. They’re there to help you learn, be more productive, and above all, enjoy the art of Mattercrafting. And this is just the beginning - we’re cooking up a ton more that we can’t wait to share with you.
Here’s to the vibe 🍻.