Introduction
In this document, we discuss the function and setup procedure of the STUDIO control panel.
NOTE: By this point, you should already have finished keying the video input. If you haven't keyed the video input yet, then we recommend proceeding to the following documents:
Studio Control Panel
The STUDIO control panel is essentially a 3D cropping tool used to establish a video crop of the tracked cameras' video input. Since tracked cameras can freely move in the studio space, cropping their video input requires additional steps compared to the Virtual Camera compounds' CROP control panels.
You can find the STUDIO control panel on the INPUTS control board.
As usual, after selecting the STUDIO control panel, you can find its adjustable parameters on the right side of the screen, on the Pin Values panel:
Setting up the Studio
Studio View
First, on the MONITOR panel, click on the Studio button to switch to Studio view mode:
You should see a virtual grid superimposed on your video input, as shown below:
NOTE: If Lens Distortion is used in the production, it will affect the Studio view. Therefore, it is optimal to first ensure that the lens distortion data has been received. Once this is confirmed, you can proceed with setting up your studio according to this document.
NOTE: In the screenshot above and in the two below, we show the default position of the grids before alignment.
The virtual helper grid has two different types:
- Wall grids: Standard grids that we align with real-world studio walls.
- Green grids: These align with the real-world studio's green screen to crop out the unnecessary parts of the video input.
Base Cam Transform
The Base Cam Transform parameters allow for aligning the orientation of all virtual Walls with our real-world physical studio.
If the tracked camera and the green screen are on the same floor (height), then do not adjust the Y parameter (which is the second box of the Pos row). Otherwise, it is suggested to use the CTRL + SHIFT key while editing the Y value for fine adjustments.
The Base Cam Transform parameters are as follows:
- Pos: Position, from left to right these are the X, Y, and Z coordinates of the virtual Walls. These parameters use meters for measurement.
- Rot: Rotation, from left to right these are the rX, rY, and rZ coordinates of the virtual Walls. These parameters use degrees for the measurement.
NOTE: You can learn more about Camera and Head Transformations here.
Walls
The Base Cam Transform is best used in conjunction with the Wall parameters. They allow for the independent positioning of the virtual Walls. These parameters use meters for measure and they are named based on the camera's perspective.
- Front Wall: The Wall in front of the camera, usually the green screen itself.
- Left Wall: The Wall to the left of the camera.
- Right Wall: The Wall to the right of the camera.
- Back Wal: The Wall behind the camera. In a bigger green screen studio, the camera may be able to move more than 180 degrees within the green screen.
- Ceiling: The ceiling of the real-world physical studio.
- Floor: If the tracked camera and the green screen are on the same floor (height), then do not adjust this parameter.
Here is our result after setting up the Walls and the Base Cam Transform:
Greens
Use the Green parameters to crop out the unnecessary parts of the video input. The Green parameters use meters for measuring, just like the Wall parameters.
- Green Front Left: adjusts the crop mask's left edge on the Front Wall.
- Green Front Right: adjusts the crop mask's right edge on the Front Wall.
- Green Left: adjusts the crop mask's left side on the Left Wall. Useful when the green screen is a corner setup.
- Green Right: adjusts the crop mask's right side on the Right Wall. Useful when the green screen is a corner setup.
- Green Top: adjusts the crop mask's height on the Front Wall.
- Green Floor: adjusts the crop mask's length on the Floor (the higher the value, the closer its edge will reach towards the camera).
Here is our result after setting up the Greens:
Wall Opacity
Lowering the Wall Opacity will make the virtual helper grid more transparent, allowing more of the video input to be visible. Conversely, increasing the Wall Opacity will make the helper grid more opaque, thereby making the video input less visible.
Show Green
The Show Green parameter shows or hides the Green area. When disabled, it allows you to clearly see the results of the crop mask, highlighting the part of the video input that you want to use.
Show Keyed
The Show Keyed parameter displays the keyed input behind the Green grids. When enabled, it helps you better define the edge of the green (crop mask). It is suggested to use this setting with the above Show Green parameter turned off.
For example, if your green screen has some wrinkles near the edge that the keyer cannot handle, these imperfections will appear as black over a white background, allowing you to easily adjust the green area to exclude them.
Show Non-Green
The Show Non-Green parameter shows or hides the non-green Walls. When disabled, it allows you to clearly see everything that has been cropped out of the video input.
Mask Edge Fade
The Mask Edge Fade parameter softens the edges of the cropped-out video. This can help prevent harsh edges when the keying is imperfect.
For instance, if you plan to retain shadows during the keying process, the Mask Edge Fade can ensure that the shadow does not have a hard cut when it extends beyond the Studio's crop mask.
Below is an example illustrating a shadow encroaching on the studio's crop mask. By using the Mask Edge Fade parameter, the shadow is seamlessly blended in the last image:
| Keyed |
Studio |
Final with 0 Mask Edge Fade |
Final with 0.25 Mask Edge Fade |
In most productions, using the parameters mentioned above is sufficient. The rest of this page discusses parameters that are used in special cases.
Mask Modes
There are three different Mask Mode that change how the Studio crop mask functions:
- Off: When the Mask Mode is set to Off, the studio mask is completely disabled. This means the video input will not be cropped.
You can use this setting, for example, when every wall in your real-world studio is a green screen. It is also useable if you are confident that the studio camera will not capture anything outside of the green screen.
Additionally, if you want to create windows into the virtual world with your green screens in the studio and are not concerned about keying affecting the entire studio, you should choose this setting. - Normal: This is the default setting and works as described in the Setting up the Studio section above.
- Window: This mode is used in productions where you want to show your real-world studio outside of the green screen in the final picture. The talent can even walk from the non-green areas into the green area.
In this mode, the non-green wall areas will display the input without any keying, while in Normal mask mode, these areas would show the virtual world without any input.
NOTE: It is also possible to move virtual objects between the green screen area and the non-green areas. In Unreal cameras, you can achieve this with the Green + AR camera. In Aximmetry-rendered scenes, this can be done by connecting the rendered AR scene's image to the Overlays pin of the camera compound.
| Studio View |
Mask Mode - Normal |
Mask Mode - Window |
|
This image is a demonstration of the Window Mask Mode's effect and is not part of an actual use case. |
Window Mask Mode Parameters
When Window Mask Mode is selected, a purple outline is generated around the green area. This purple area must overlap the edges of your green screen.
NOTE: The purple area is not part of the billboard but is still keyed to provide a smooth transition between the green screen and the studio.
Two additional parameters also appear to customize the purple area:
- Window Size: Adjusts the size of the purple area.
- Window Edge Fade: Similar to Mask Edge Fade, this parameter creates a smoothing between the keyed and non-keyed parts of the image. It also smooths out the Despill of the keying.
Studio Models
Studio Models can be used when the Studio control panel is not enough to crop your green.
These models need to be connected to the object reference pins of the Tracked Camera compounds:
NOTE: The examples below will use basic primitive shape modules of the Flow Editor. If you want to use your own models and you are not sure how to import models, then follow this document: Importing a 3D model
Green Model
You can specify extra green areas by importing and connecting 3D models to the Green Model pin.
These models for example can specify the boundaries of an irregular green wall.
For example, here a Rectangle body module is connected to the Green Model pin.
By rotating the Rectangle it is possible to make the green floor diagonal:
NOTE: The Rectangle module is a basic primitive shape inside the Flow Editor. The Scene Node module is used to rotate and position the Rectangle model.
Garbage Mask
You can specify extra non-green areas by connecting 3D models to the Garbage Mask pin.
These models for example can mask out a studio lamp in front of a green or other studio equipment that may go over the edges of your green screen from certain angles of your studio camera.
Here a studio lamp is masked out from the green. The green area was set in the Studio control panel and the mask was created by connecting Box body modules to the Garbage Mask pin.
| Without Garbage Mask |
With Garbage Masks |
NOTE: Show Green and Show Non-Green pins also affect these models' visibility and can help greatly when placing these models.
Holdout Mask
These models prevent the keyer from keying out specific areas. This is useful for avoiding keying out a prop that has a similar color to the green screen.
For example, consider a chair that has a sticker with "Chair" written on it:
Because the sticker is green, it gets keyed out, causing the keyer to remove part of the chair. Using a Holdout Mask over the sticker, the keyer will ignore the area defined by the Holdout Mask's model, and the chair remains whole:
| Without Holdout Mask |
With Holdout Mask |
Window Model
The Window Model can be used to define an additional purple transition area when in Window Mask Mode.
NOTE: If you are using Green Models, you will have to define the purple transition area around them with the Window Model.
Additional Pins
Show Floor Zero
This parameter highlights the zero point of the floor. A 1 by 1 meter area is highlighted around the studio's 0x0 coordinates:
Show Cameras
This parameter displays the other INPUTs' camera in the studio.
Only the cameras with their INPUT turned on are shown.
For example, if you are in the studio monitor of INPUT 1 and INPUT 2 is also turned on:
Then you can see the location of INPUT 2's camera in the studio view. If INPUT 2 happens to be the selected camera, it will be highlighted by red recording indicators:
Show Cam Heights
This parameter displays the height of each camera with a green-blue column and shows the exact height from the floor in meters above each camera.
It also highlights the camera's x and y positions on the floor with white lines. This can be helpful when comparing the positions of different cameras, especially when setting up and comparing two different tracking solutions.
NOTE: The green and blue colors of the column change every 1 meter height.
Show Cam Frustums
This parameter displays the camera's frustum from the camera's sensor. This can be helpful for visualizing zoom data.
NOTE: The higher the camera's focal length (measured in mm), the more zoomed in you are. Consequently, the camera's perspective will be narrower (FoV), and the frustum's area will be smaller.
Talent Trk
These pins are used when the tracking system for talent tracking differs from the camera tracking system, and aligning their coordinate systems is not possible through any other method.
Show Talent Trk
This parameter toggles the display of the Talent Tracking's position in the Studio view.
NOTE: The exact position is marked as a point, while the accompanying text is the Billboard's ID:
Show Talent Trk Area
This parameter toggles the display of the Talent Trk Area in the Studio view. NOTE: This display is influenced by the Wall Opacity parameter.
Talent Trk Area Size
This parameter defines the size of the Talent Trk Area. It does not affect the actual talent tracking. It is intended to help in the alignment of the Talent Trk Transf.
Use Talent Trk Transf
Enables the usage of the Talent Trk Transf value to align the two tracking systems.
Talent Trk Transf
This parameter allows the alignment of the Talent Tracking device's coordinate system with the camera's coordinate system.
Markers Visible
When the In Studio and Scene option is selected for the Markers Visible parameter, cameras are also displayed over the scene, a floor is shown starting from the zero position, and additional Markers can be defined with the Scene Marker Amount parameter.
With the In Studio and Scene option, two new parameters appear:
- Scene Floor Size: This parameter sets the size of the floor over the scene.
- Scene Marker Amount: This parameter allows you to add Markers over the scene.
Studio Marker Amount and Scene Marker Amount
Changing the Marker Amount from 0 reveals additional parameters for configuring markers in either the Studio or Scene. The number you select determines the total markers you can configure.
Each marker starts with the Marker Type parameter, allowing you to choose from eight different types. These markers can serve various purposes, for example, leaving notes in your studio or scene environment.
