Introduction
This document gives a general overview of Gaussian Splats, including their key features and characteristics, and describes how you can use them as graphical content in a virtual production.
What Are 3D Gaussian Splats?
In the world of virtual production, a Gaussian Splat is typically a representation of a 3D scene as a dense collection of small, semi-transparent, oriented ellipsoids called Gaussians, instead of a traditionally used polygon mesh. Each Gaussian stores a position, a shape defined by scale and orientation, a color that can vary by viewing angle, and an opacity value.
A third-party Gaussian generator software creates a Gaussian Splat from a set of photos or a video of a real environment or object. It studies the images and works out millions of Gaussians that, together, reproduce what the camera saw. This process is often called splat training or reconstruction.
To display a Gaussian Splat, the software projects the Gaussians onto the screen and blends them together, a process called splatting. Splatting is a technique fast enough to allow for real-time playback, while still offering a photorealistic look.
How 3D Gaussian Splats Can Be Used for Virtual Production
On a higher level, a Gaussian Splat is a 3D asset, so you can use it as a source of graphical content in the same way as a traditional 3D model. You can place a 3D Gaussian Splat of a real location or object into a virtual production scene and use it as a background, an environment, or a set element, for green screen production, augmented reality, or LED wall production.
NOTE: There are 2D, 3D and even 4D (animated) Gaussian Splats. For virtual production, we typically use 3D Gaussian Splats, hence we focus on this type of Gaussian Splats.
Aximmetry offers two render options for virtual production scenes: Aximmetry's native render engine and Unreal Engine:
Aximmetry Native Engine
Aximmetry's Native Engine currently does not support the rendering of Gaussian Splats.
Unreal Engine
Unreal Engine renders Gaussian Splats through third-party plugins that add Gaussian Splat support to the Unreal Editor. Because AX Scene Editor is a customized version of the Unreal Editor, a plugin that renders Gaussian Splats in the standard Unreal Editor can also work in AX Scene Editor, provided the plugin's compatible with the Aximmetry Composer render pipeline.
NOTE: Plugin behavior varies. Some plugins render Gaussian Splats correctly in Live Sync mode but do not support the cooking process that AX Scene Editor uses to produce a runtime project. Test a plugin's compatibility with your Aximmetry version before you rely on it for production.
NOTE: For information about adding third-party plugins to AX Scene Editor, see:
How to Install Third-Party Code Plugins for AX Scene Editor
NOTE: For information about specific 3D Gaussian Splat plugins and how they are used with Aximmetry, refer to the following documents:
Using 3D Gaussian Splats with XYN Spatial Renderer Plugin
