Using the 3D Clean Plate Generator

Introduction

In this documentation, we will show you how to use the 3D Clean Plate feature to achieve better keying with tracked cameras. 
The 3D Clean Plate generator captures the texture of your green background and applies it to a 3D model of the green background that moves together with the real one allowing you to create a Clean Plate that follows your camera movements.

Requirements

To take advantage of the 3D Clean Plate feature there are a few requirements.

• You must have an Aximmetry Broadcast Edition to use this feature.
• You must have a reliable and accurate tracking system.
• You must have an accurate 3D model of your green background.

Model Requirements

There are a few methods to acquire a model of your green background.

• Use the model that was used to build your studio
• Create a model based on measurements.
• Use a 3D scanner tool/application - please note that these models usually require cleanup after the scanning has been done as they usually contain unnecessary parts of your studio that can be removed.

Your model must meet some very important criteriums to be used with the 3D Clean Plate Generator.
These criteria are:

• You should try to match your real studio in size as much as possible but if it is a few centimeters off it is okay.
• The model's pivot point must be on the floor at 0 height.
• The model must not have overlapping faces.
• The model must be properly UV unwrapped. There should not be overlapping vertices in the UV map.
•  Aximmetry must be able to triangulate the model.
• The model must not have any material applied to it.
•  It must be in either of the following file formats: .fbx, .obj or .dae.

Aximmetry Setup

This documentation assumes that you are familiar with setting up a virtual studio with a tracked camera and using Aximmetry's own keyer.
If not please click on the links above to learn more.

After loading your scene you should see something like this in the flow editor:
SE:

DE:

Importing the Studio Model

NOTE: From this point, we are only going to show the SE version as the workflow is the same with DE as well.

You should import your 3D model into Aximmetry by drag-n-dropping it into the flow editor.

Set up the connections like so:
 

Important: The 3D model must not have any material applied to it.

To make sure that it does not have right-click on the model's compound and select Set Importer Options...
In the pop-up window untick Import materials and click OK.:

Aligning the Model with the Real-World Studio

Go to the INPUTS control board and select the Keyed in the MONITOR panel.
This will also select the KEYER panel.

In the KEYER panel's property editor change the KEYER TYPE to CLEAN PLATE 3D GENERATOR

By default, the Monitor Mode is set to Blended Cropped which allows you to see both the camera image and the 3D model of your studio with a semi-transparent purple shader.

NOTE: sometimes you need to press the Move Model in front of Camera button to be able to see it.
This is because the origins of the tracking system and the 3D model do not align completely.

Meaning of the other monitor modes:

1. Original: Displays the raw camera image without any modifications or overlays.
   
2. Clean Plate: In this mode only the Clean Plate is visible either with a purple shader or with the captured texture.
3. Blended: Overlays the Clean Plate onto the raw camera image.
4. Blended Cropped: Overlays the Clean Plate onto the raw camera image and darkens the cropped edges, which have been trimmed by the Crop Fraction.
5. Blended Wireframe: Displays a wireframe representation of the 3D model of the Clean Plate, showing the structure and geometry of the model.
   

Using the Transformation property position the 3D model so that it matches your real-world studio.
You can use the Blended Wireframe monitor mode to see the 3D model as a grid if you find it easier to align this way.

Important: The height of the 3D model should not be changed.

Capturing the Clean Plate

After aligning your studio you can start capturing the 3D Clean Plate.

To capture the 3D Clean Plate you need to look at your green screen from an arbitrary point of view. Then press Capture in the KEYER panel:

This will capture that part of the green screen:

You should then rotate your camera a bit and click Capture again, this will capture another part of the green screen.

NOTE: Only the inner portion of the captured image is used for creating the Clean Plate's texture. This area is defined by the Crop Fraction. The Crop Fraction helps to prevent artifacts at the edges caused by the camera's lens, which can result in an imperfect texture.

Using the Clean Plate

Once you have finished creating the Clean Plate, switch back to the Advanced B Keyer Type.
Turn on Use Clean Plate to utilize the Clean Plate's image instead of the Background Color: