Introduction
In this documentation, we will show you how to use the 3D Clean Plate feature to achieve better keying with tracked cameras.
The 3D Clean Plate generator captures the texture of your green background and applies it to a 3D model of the green background that moves together with the real one allowing you to create a Clean Plate that follows your camera movements.
Requirements
To take advantage of the 3D Clean Plate feature there are a few requirements.
- First of all, you must have a reliable and accurate tracking system.
- You must have an accurate 3D model of your green background.
Model Requirements
There are a few methods to acquire a model of your green background.
- Use the model that was used to build your studio
- Create a model based on measurements.
- Use a 3D scanner tool/application - please note that these models usually require cleanup after the scanning has been done as they usually contain unnecessary parts of your studio that can be removed.
Your model must meet some very important criteriums to be used with the 3D Clean Plate Generator.
These criteria are:
- You should try to match your real studio in size as much as possible but if it is a few centimeters off it is okay.
- The model's pivot point must be on the floor at 0 height.
- The model must not have overlapping faces.
- The model must be properly UV unwrapped. There should not be overlapping vertices in the UV map.
- Aximmetry must be able to triangulate the model.
- The model must not have any material applied to it.
- It must be in either of the following file formats: .fbx, .obj or .dae.
Aximmetry Setup
This documentation assumes that you are familiar with setting up a virtual studio with a tracked camera and using Aximmetry's own keyer.
If not please click on the links above to learn more.
After loading your scene you should see something like this in the flow editor:
SE:
DE:
Importing the Studio Model
NOTE: From this point, we are only going to show the SE version as the workflow is the same with DE as well.
You should import your 3D model into Aximmetry by drag-n-dropping it into the flow editor.
Set up the connections like so:
Important: The 3D model must not have any material applied to it.
To make sure that it does not have right-click on the model's compound and select Set Importer Options...
In the pop-up window untick Import materials and click OK.:
Aligning the Model with the Real-World Studio
Go to the INPUTS control board and select the Keyed in the MONITOR N module.
This will also select the KEYER N module.
In the KEYER N modules property editor change the KEYER TYPE to CLEAN PLATE 3D GENERATOR
By default, the Monitor Mode is set to Blended which allows you to see both the camera image and the 3D model of your studio with a semi-transparent purple shader.
NOTE: sometimes you need to press the Move Model in front of Camera button to be able to see it.
This is because the origins of the tracking system and the 3D model do not align completely.
Meaning of the other monitor modes:
- Original - raw camera image
- Clean Plate - in this mode only the Clean Plate is visible. Either with a purple shader or with the captured texture.
- Blended - Clean Plate overlayed on the raw camera image
Using the Transformation property position the 3D model so that it matches your real-world studio.
You can use the Wireframe mode to see the 3D model as a grid if you find it easier to align this way.
Important: The height of the 3D model should not be changed.
Capturing the Clean Plate
After aligning your studio you can start capturing the 3D Clean Plate.
To capture the 3D Clean Plate you need to look at your green screen from an arbitrary point of view. Then press Capture in the KEYER N module:
This will capture that part of the green screen:
You should then rotate your camera a bit and click Capture again, this will capture another part of the green screen.
Do this until you cover all used parts of your green screen.
NOTE: It's not a problem that you do not cover 100% of the greenscreen, you should only cover the parts that are visible in the camera during a show.
In the end, you should have something like this:
If you need to do the capture again for some reason you can always press Reset in the property editor to start again.
When you have to Capture it Again
Changing any of the following settings would require you to capture the 3D Clean Plate again:
- Lighting setup in the studio
- Camera ISO
- Camera aperture
- Camera shutter speed
Using the Captured Texture
Set the KEYER TYPE to ADVANCED B in the Pin Values panel and turn on Use Clean Plate:
If you have a keyer that is already set up to your green background it is advised to reset the Low Cut and High Cut values.
This is because the 3D Clean Plate will allow you to use much lower or even 0 Low Cut and 1 High Cut.
From this point onward, the keyer can be used as usual (see: Keying).
NOTE: We recommend continuing to read the document, and navigating back to the Tracked Camera Workflow using the links at the bottom.
Saving the Texture
After the capture has been done you can click on the Save Texture trigger (while in 3D Clean Plate Generator Keyer Type) to save the texture into a file.
This will allow you to use the Clean Plate texture later without having to capture it again.
After the texture has been saved it will be automatically loaded into the Texture From File property.
Limitations
NOTE: Even with these limitations you should get better results than without using a 3D Clean Plate.
Hard Edges in the Green Screen
If your green screen is made of parts that are very different in color you may experience that these edges are visible in the keying.
This is because the 3D model and the real studio do not move completely in sync.
Possible causes of this:
- Your tracking system is not 100% accurate.
- Your calibration is not 100% accurate.
- Your model is not accurate enough.
Camera Vignetting
Every camera has imperfections, vignetting is one of them. This makes the sides of the captured image darker than the center.
We try to correct it using our vignette correction feature. You can read about it here.
If you manage to create a good vignette correction your Clean Plate will be much more homogenous.
This will result in a nicer keying in the end.
NOTE: An incorrect vignette correction can even make the capture worse.
However, creating a good vignette correction for a camera is not an easy task.
Even with vignette correction, you may find darker or lighter spots in your clean plate like so:
We try to eliminate this with the Crop Fraction property. We reckon the default 0.3 value is good enough but in some cases, it may not be enough.
In this case, you can try to increase the Crop Fraction. This will crop the edges of the camera image and make it only capture the center of it into the Clean Plate texture.
You will need to capture more often but this will eliminate the vignetting of the camera more. We suggest not going over the 0.5 value.
NOTE: With good vignette correction it is possible to reduce the Crop Fraction value.
Continue to the Next Step
Once you finished setting up your 3D Clean Plate, you can move on to the next step.
Please continue here if you are following the Tracked Camera Workflow:
