Aximmetry DE Scene Setup (Green Screen)

NOTE: This document assumes the reader has a basic understanding of Aximmetry DE and familiarity with camera compounds. If these concepts are new to you, please consult the documentation for Introduction to Green Screen Production and Preparing the Unreal Project before proceeding.

Basic Features

Setting up a virtual camera and billboard rendering in Unreal Engine can be a complex process. To greatly simplify this, we have developed Aximmetry Camera blueprints in Unreal Engine.

It provides the following features:

• Aximmetry Camera: Allows Aximmetry to control the position, orientation, Field Of View (FOV/zoom), and Depth Of Field (DOF/focus) of the virtual camera. It is based on the Cine Camera, enabling the addition of post-process effects, etc.
  
• Billboard Rendering: This aspect provides the ability to have up to three billboards visible in a scene. Specifically, a virtual camera can display billboards from three different physical camera inputs, while a Tracked camera can show three billboards from one camera input.
  • Option to include shadows
  • Can be affected by scene lights
  • Ability to be influenced by Unreal's tone mapper
  • Enhanced image quality and sharpness

Billboards

Billboard Settings

When using the Tracked Green camera the Use Billboards has to be turned on for the Billboard to interact with the virtual world.

To designate whether Aximmetry or Unreal Engine is rendering the billboard, the Allow Virtuals setting can be changed. Documentation regarding this feature is arranged according to the Virtual or Tracked camera categories:

• For the Virtual Camera in the Billboards in Virtual Camera Compounds with Aximmetry DE (Unreal Engine Rendered) section.
  
• For the Tracked Camera in the Allow Virtuals section.

For comprehensive details on configuring billboard settings based on the camera type in use, documentation is organized per the following pages:

• For the Virtual Camera at Setting Up Billboards in Virtual Camera Compounds.
  
• For the Tracked Camera at Tracked Camera Billboards: Placement.

Translucent Objects at the Billboard

Billboards are rendered in the After DOF Translucency pass of Unreal. This is the default Translucency pass for new materials in Unreal.
All translucent materials should be set to After DOF translucent pass or they might be rendered over or behind the billboard no matter their position:

To change the Translucency pass to the correct one:
Go to the Material Editor window by double-clicking the translucent material of the object in the Content Browser.
In the Details panel, go to the Translucency / Advanced section and set the Translucency Pass to After DOF:

NOTE: Using several overlapping translucent materials can significantly increase the processing load on Unreal Engine hardware in comparison to other material types.

Translucent Clipping

If your scene contains translucent objects that might clip through the billboard, you should be aware of this constraint in Unreal Engine:
Unreal Engine determines clipping between translucent objects based on actor position and not on absolute pixel position, this can cause abrupt transitions between being in front of or behind each other:

There are several methods to determine the rendering order of translucent objects. For example, adjusting an actor's Translucency Sort Priority can influence this order.
For additional information about handling transparency and translucency sort priority in Unreal, please refer to the documentation from Unreal.

Objects in Front of the Billboard

When positioning objects in front of the billboard and using the Allow Virtuals setting, certain steps must be taken to prevent visual artifacts. This necessity arises because Unreal Engine employs temporal antialiasing techniques, specifically Temporal Anti-Aliasing (TAA) and Temporal Super-Resolution (TSR), to smooth the edges of objects. These techniques, however, can cause the billboard to appear very blurry. To address this issue, we modified these antialiasing methods. Our modified versions utilize CustomDepth to detect objects placed in front of the billboard, thereby ensuring the accurate handling of edges and translucency. To provide this CustomDepth information, you have to mark the objects that can occlude the billboard with a stencil value.

If the object is opaque and you are using TAA:

• Select the object.
  • In the Detail panel, go to the Rendering / Advanced section.
  • Enable Render CustomDepth Pass and Set CustomDepth Stencil Value to 16:
    
    

Not doing this results in aliasing and jittering at the object's edges, for example:

If the object is translucent and you are using TAA or TSR:

• Select the object.
  • In the Detail panel, go to the Rendering / Advanced section.
    
  • Enable Render CustomDepth Pass.
  • Set CustomDepth Stencil Value to 32.
• Go to the Material Editor window by double-clicking the translucent material of the object in the Content Browser.
  • In the Details panel, go to the Material / Advanced section and enable Allow Custom Depth Writes.
    
  • In the Details panel, go to the Translucency / Advanced section and set the Opacity Mask Clip Value to 0.

Not doing this results in aliasing and jittering at the edge of the object, for example:

NOTE: Due to this, Unreal Engine material solutions that significantly depend on the use of Allow Custom Depth Writes may exhibit artifacts at the billboard.

TSR

In Unreal Editor for Aximmetry, the default antialiasing method is TAA because we found that TSR can cause rendering errors on the billboard if an opaque object occludes it.
If this is not an issue in your use case, you can switch to TSR.

Nanite virtualized geometry

Objects with Nanite mesh can not be in front of billboards when Allow Virtuals is turned Off or Light Wrap is turned On. Otherwise, the billboard will be rendered with artifacts, including the billboard being incorrectly rendered in front of the Nanite mesh object.

NOTE: Nanite is intended for use in scenes featuring a high count of objects in the background.

Motion Blur

Motion blur can make scenes unrealistic and cause artifacts, for example at fast camera movements.

If you want to disable Motion Blur:

• Go to Edit / Project Settings - Engine / Rendering / Default Settings
• Untick Motion Blur

Motion Blur - On:

NOTE: Movement of the talent on the billboard (reflected as changes in the billboard's texture) will not produce motion blur, even when the Motion Blur is activated.

Motion Blur - Off:

Camera Focus

When Allow Virtuals is turned on, the focus affects the billboard.

When changing focus in Aximmetry, the Aximmetry camera changes the Cinematic Depth of Field (DOF) method's manual focus distance.

The console variable r.Translucency.AutoBeforeDOF influences when the focus distance starts to affect the billboard and other translucent objects. Its default value is set to 0.5. Assigning a value of 1 indicates that translucent objects remain unaffected by focus. Thus, by configuring this variable to 1, you can turn off the focus on the billboard.

Ray Tracing

The Unreal Engine offers support for ray tracing, which includes a variety of features that can be individually toggled on or off. For more detailed information, please refer to the associated Unreal documentation.

It's important to understand that all ray tracing capabilities require significant computational resources, even with modern hardware. Thus, it's advisable to start with all ray-tracing features turned off, then selectively enable those that significantly enhance your specific scene. You can find detailed instructions on adjusting these settings in the relevant Unreal documentation.

While the camera is compatible with most ray tracing features, there are certain limitations within the Unreal Engine that users should be aware of:

• Ray Tracing Global Illumination is fully supported.
• Ray Tracing Reflections and Ray Tracing Shadows are supported but come with specific restrictions. These features, along with other reflection and shadow rendering methods, will be described in greater detail below.
  
• Ray Tracing Ambient Occlusion and Ray Tracing Translucency are not supported.

Shadows

The Green cameras provide support for both rasterized and ray-traced Billboard shadows.
However, it is important to note that Billboards do not support Distance Field Shadows.

Ray-traced shadows

Ray-traced shadows can be activated via: Project Settings - Engine / Rendering / Hardware Ray Tracing / Ray Traced Shadows.
Also, ensure that: Project Settings - Engine / Rendering / Hardware Ray Tracing / Support Hardware Ray Tracing is enabled.

Limitations

Cast Ray Tracing Shadows must be set to Use Project Setting for all light sources.
Changing Ray Traced Shadows during runtime is not supported.

Reflections

The Green cameras support all reflection methods that are available in the Unreal Engine for rendering billboard reflections. However, there are some limitations due to how reflection rendering and ray-tracing work in the engine.
This section describes each available reflection method's properties, requirements, and limitations.

Key Variables

To understand how various features interact with reflection rendering, the following variables need to be considered or changed:

• In the Unreal Editor for Aximmetry:
  • Project Settings - Engine / Rendering / Global Illumination / Dynamic Global Illumination Method
  • Project Settings - Engine / Rendering / Reflections / Reflection Method
  • Project Settings - Engine / Rendering / Hardware Ray Tracing / Support Hardware Ray Tracing and Ray Traced Shadows
    
  • Project Settings - Engine / Rendering / Lumen / Use Hardware Ray Tracing when available and Ray Lighting Mode
    
• In Aximmetry:
  • Virtual Camera - BILLBOARDS control board's Billboard panel: Auto Mirror Offset, Mirror Offset, Mirror Feet Blur, Feet Blur Offset, Cast Shadows, Shadow Depth, Shadow Offset, Shadow Rotation, Lit, Render To Depth and Ray Traced Reflection Intensity.
    
  • Tracked Camera - TRK INPUTS control board's Billboard panel: Mirror Feet Blur, Blur Depth, Blur Amount, Mirror Offset Z, Cast Shadows, Shadow Depth, Shadow Offset, Shadow Rotation, Lit, Render To Depth and Ray Traced Reflection Intensity.
    

List of Reflection Methods

1. Planar Reflections
   

   High-quality reflections. Semi-transparent areas are reflected correctly.

   • Reflection Method must be None or Screen Space.
     
   • Render To Depth must be Off.
     
     
   • A Planar Reflection actor must be added to the scene.

2. Lumen Reflections

   Average quality reflections. Semi-transparent areas are not reflected.
   
   
   • Reflection Method must be Lumen.
   • Support Hardware Ray Tracing must be enabled.
   • Use Hardware Ray Tracing when available must be enabled.
   • Ray Lighting Mode must be Hit Lighting for Reflections.
   • Render To Depth must be On.
     
     
   • When a Virtual Camera is used: the Auto Mirror Offset, Mirror Offset, Mirror Feet Blur, and Feet Blur Offset billboard properties have no effect.
   • When a Tracked Camera is used: the Mirror Feet Blur, Blur Depth, Blur Amount, and Mirror Offset Z billboard properties have no effect.
     
     
   • When Lit is Off, the brightness of the reflection can be different from the brightness of the billboard. This applies only to the reflection of offscreen parts of the billboard.
     
   • When Ray Traced Shadows are enabled, Cast Shadows is On: the Shadow Depth, Shadow Offset, and Shadow Rotation billboard properties affect the reflection. This applies only to the reflection of offscreen parts of the billboard.
   • When Ray Traced Shadows are enabled, Cast Shadows is On, the Shadow Depth is greater than 0, and Lit is On: lighting can be incorrect on the reflection. This applies only to the reflection of offscreen parts of the billboard.
     
     
   • Increase the Ray Traced Reflection Intensity until the brightness of the reflection matches the brightness of the billboard.
     

3. Screen Space Reflections

   Low-quality reflections. Semi-transparent areas are not reflected.

   • Reflection Method must be Screen Space.
   • Render To Depth must be On.
     
     
   • When a Virtual Camera is used: the Auto Mirror Offset, Mirror Offset, Mirror Feet Blur, and Feet Blur Offset billboard properties have no effect.
   • When a Tracked Camera is used: the Mirror Feet Blur, Blur Depth, Blur Amount, and Mirror Offset Z billboard properties have no effect.
     
     
   • A Planar Reflection actor must be added to the scene.
     

4. Standalone Ray Traced Reflections

   Average quality reflections. Semi-transparent areas are not reflected.

   • Reflection Method must be Standalone Ray Traced.
   • Support Hardware Ray Tracing must be enabled.
   • Render To Depth must be On.
     
     
   • When a Virtual Camera is used: the Auto Mirror Offset, Mirror Offset, Mirror Feet Blur, and Feet Blur Offset billboard properties have no effect.
   • When a Tracked Camera is used: the Mirror Feet Blur, Blur Depth, Blur Amount, and Mirror Offset Z billboard properties have no effect.
     
     
   • When Lit is On, the brightness of the reflection can be different from the brightness of the billboard.
   • When Ray Traced Shadows are enabled and Cast Shadows is On: the Shadow Depth, Shadow Offset, and Shadow Rotation billboard properties affect the reflection.
     
   • When Ray Traced Shadows are enabled, Cast Shadows is On, the Shadow Depth is greater than 0, and Lit is On: lighting can be incorrect on the reflection.
     
     
   • Increase the Ray Traced Reflection Intensity until the brightness of the reflection matches the brightness of the billboard.
     

Limitations

• • Overriding the Reflection Method via a Post Process Volume or camera post-process settings is not supported.
  • Changing the Reflection Method during runtime is not supported.
  • Increasing Ray Traced Reflection Intensity can cause a noticeable glow around the billboard if the Dynamic Global Illumination Method is set to Lumen or Standalone Ray Traced. If this happens, try to decrease the Ray Traced Reflection Intensity until the glow is not noticeable anymore and the reflection is not too dark either.