Studio Control Panel

Back to Inputs (Tracked Camera)

Introduction

In this document, we discuss the function and setup procedure of the STUDIO control panel.

Studio Control Panel

The STUDIO control panel is essentially a 3D cropping tool used to establish a video crop of the tracked cameras' video input. Since tracked cameras can freely move in the studio space, cropping their video input requires additional steps compared to the Virtual Camera compounds' CROP control panels.

You can find the STUDIO control panel on the INPUTS control board.

As usual, after selecting the STUDIO control panel, you can find its adjustable parameters on the right side of the screen, on the Pin Values panel:

Studio View

To visualize the Studio and its parameters, click the Studio button on the MONITOR panel to switch to Studio view mode:

You should see a virtual grid superimposed on your video input, as shown below:

To actually see the green part and the back wall, you may need to rotate your real-world camera or use the Rotate to Camera feature.

NOTE: If Lens Distortion is used in the production, it will affect the Studio view. Therefore, it is optimal to first ensure that the lens distortion data has been received (see link). Once this is confirmed, you can proceed with setting up your studio according to this document.
NOTE: In the screenshots above and below, we show the default position of the grids before alignment.

The virtual helper grid has two different types:

• Non-Green Area (Wall Grids)

  Standard grids that we align with real-world studio walls.
  

• Green Area (Green Grids)

  These align with the real-world studio's green screen to crop out the unnecessary parts of the video input.
  

Different Monitor mode views after setting up the Studio Mask:

		Keyer's Monitor Mode set to Input

Mask Modes

There are three different Mask Modes that change how the Studio crop mask functions:

• Off: When the Mask Mode is set to Off, the studio mask is completely disabled. This means the video input will not be cropped.
  You can use this setting, for example, when every wall in your real-world studio is a green screen. It is also usable if you are confident that the studio camera will not capture anything outside of the green screen.
  Additionally, if you want to create windows into the virtual world with your green screens in the studio and are not concerned about keying affecting the entire studio, you should choose this setting.
  
• Normal: This is the default setting and functions as described throughout the rest of this page. In this mode, anything outside the green area (grids) will always display the virtual world, with no camera input or billboard visible in those regions.
• Window: This mode is used in productions where you want to show your real-world studio outside of the green screen in the final picture. The talent can even walk from the non-green areas into the green area.
  In this mode, the areas outside the green area will display the input without any keying, while in Normal mask mode, these areas would show the virtual world without any input.
  
  The point at which the transition occurs between the keyed area and the real world is determined by the Margin Size parameter, or by the Margin Model when using Custom Model Green Area Mode. The smoothness of the transition is controlled by the Window Edge Fade parameter.
  It is also possible to move virtual objects between the green screen area and the non-green areas. In Unreal cameras, you can achieve this with the Green + AR camera. In Aximmetry-rendered scenes, this can be done by connecting the rendered AR scene's image to the Overlays pin of the camera compound.

Difference between Normal and Window Mask Mode while using Billboards:

Mask Mode: Normal	Mask Mode: Window	Mask Mode: Window   + AR

Green Area Modes

There are three different Green Area Modes that determine how you define the green area of your studio:

• Cube: This is the legacy mode. The green area is defined as the inside of a cube that simulates a cube-shaped room. You can specify how much green is on each wall of the cube. The cube has hard, diagonal edges and cannot define a curved green area. This mode cannot be used in 3D Cleanplate workflows.
• Curved: This is the default setting. It is designed for commonly used curved green screen setups, such as Plane, L-Shape, Corners, and U-Shape configurations. First, you determine the position of the green area’s back relative to the tracking system’s zero point. Then, you set the shape and size of the green area. This mode supports the use of 3D Cleanplate.
• Custom Model: This mode is fully customizable through custom 3D models. To define the green area, you must provide a 3D model. This approach typically requires some experience with 3D modeling and is therefore considered more complex than using Curved mode. This mode also supports the use of 3D Cleanplate.

Curved - Green Area Mode

The parameters for the Curved Green Area Mode are organized in a logical sequence, making it quick and easy to define your setup if you follow them in order. You will likely need to adjust most parameters to accurately configure your green area. All distance measurements are in meters, and all rotations are in degrees.

Back Wall

To define the Curved Green Area, you first need to determine the location of the green's back. This is achieved by adjusting the following parameters.

• Vertical Rotation: This parameter defines the rotation of the back wall relative to the zero point of your tracking system (origin point).
  When setting this value, focus only on making sure that the virtual back wall appears visually parallel to your actual green screen back wall. Do not worry if the green screen’s location appears far off at this stage; later parameters will allow you to accurately adjust its exact position.
• Rotate to Camera: This feature allows you to automatically set the Vertical Rotation by copying the current camera’s rotation in your studio and applying it to the Vertical Rotation parameter. To use this feature accurately, rotate your camera so it is directly facing (looking at) the green's back wall.
• Back Wall Distance: This parameter specifies the distance between the back wall and the zero point (origin) of your tracking system. If you are unable to accurately measure the distance from the origin point, you can use the helper graphics to estimate it by comparing their size to objects at your green screen back wall or to the entire green screen wall itself. In the helper graphics of the studio view, the smallest rectangle represents 10 x 10 cm, while the larger rectangle represents 50 x 50 cm:
  
  
• Detect Back Wall: This feature allows you to automatically set the Back Wall Distance. To use this, you have to place an ArUco marker with ID 70 vertically on the back wall. The marker should be generated from the 6x6 standard dictionary, with a printed size of 180 mm (18 x 18 cm). You can generate the marker here: https://chev.me/arucogen.

Here is our result after setting up the Back Wall:

Green Area

Use the parameters of the Green Area to define the exact shape and boundaries of your green screen area. Anything outside the defined Green Area will be cropped from the input (real-world camera) when using the default normal mask mode.

• Right End: Specifies the right boundary of the green area. Set this so that the right end of the virtual green stops before the edge of your physical green screen. If the ends of your green screen are of poor quality, it is best to set the boundary before those areas.
• Detect Right End: This feature allows you to automatically set the Right End. To use this, place an ArUco marker with ID 70 vertically on the right side of the back wall. The marker should be generated from the 6x6 standard dictionary, with a printed size of 180 mm (18 x 18 cm). You can generate the marker here: https://chev.me/arucogen.
• Left End: Specifies the left boundary of the green area. Set this so that the left end of the virtual green stops before the edge of your physical green screen. As with the right end, if the ends are of poor quality, it is best to define the boundary before the poor quality.
• Detect Left End: This feature allows you to automatically set the Left End. To use this, place an ArUco marker with ID 70 vertically on the left side of the back wall. The marker should be generated from the 6x6 standard dictionary, with a printed size of 180 mm (18 x 18 cm). You can generate the marker here: https://chev.me/arucogen.
• Height: Specifies the height of your green screen. Set this so that the top of the virtual green stops before the edge of your physical green screen. If the top parts of your green screen are of poor quality, it is best to set the boundary before those areas.
  NOTE: If you are confident that you have accurately measured your green screen’s height (for example, by using a tape measure), but still notice a mismatch between the size of the virtual graphics pattern and the real-world green screen, you may have set the Back Wall Distance incorrectly.
• Shape: There are five different shapes available: Plane, L Shape, Left Corner, Right Corner, and U Shape. These progress from the simplest green area shape (Plane) to the most complex (U Shape). Changing the shape will determine how many Curve parameters are available.
• Depth: Defines the length of the green area’s floor. This measurement extends from the back wall toward the center of your tracking system (origin point). Set this so that the end of the virtual green stops before the edge of your physical green screen. As with the previous parameters, if the ends of the green are of poor quality, it is best to define the boundary before the poor quality.
• Bottom: Available only in a Plane Shape. Sets the plane's starting point from the floor.
• Edge: Can be set to 1 or 2 curves. The curve defines the transition between the floor and the walls. If set to 2 curves, you can define a more irregular curvature between the floor and walls. In 2 curves mode, the first curve value determines the length of the curvature on the floor, while the second value determines the length along the wall.
• Curve ...: Depending on the selected Shape, there may be multiple curve parameters, one for each wall, or none if you select the Plane Shape. This parameter determines the extent of curvature between the floor and the individual walls. The values are measured in meters and define how much the curved space occupies on either the wall or the floor (or both when using 1 Curve Edge). If set to 0, there is no curvature, resulting in a right-angle corner between the wall and floor.

Here is our result after setting up the Green Area:

Margin Size

Adjusts the size of the black area surrounding the green screen area. You can think of this as an extension of the green area.
This extended area is used by two features: 3D Clean Plate and Window Mask Mode.

When Window Mask Mode is selected, this black margin must overlap the edges of your physical green screen. The black area itself will not appear as part of the billboard, but it will still be keyed to provide a smooth transition between the green screen and the studio environment.

Monitor Mode

There are three options available to change the view in the Studio monitor using the Monitor Mode parameter:

• Blended: This is the default option. It displays the virtual studio's walls (green and non-green areas) over the image from the real-world camera input. The opacity of the studio walls can be adjusted with the Walls Opacity parameter.
  
• Blended Wireframe: In addition to the Blended view, this mode also overlays the wireframe (vertices) of the green area's model. This can be helpful, for example, for accurately defining the Curve parameters of the green screen area.
• Keyed and Masked: This mode displays only the real-world camera input after it has been keyed and masked based on the green area. It is useful for evaluating the results of the keying process, for example, if parts of the green screen are not properly keyed and you want to hide those areas by making the Green Area smaller.

Show Schematics

There are four options available to add helper schematics to the Studio monitor's view using the Show Schematics parameter:

• None: This is the default option. No schematics are shown.
• Sizes: Displays the various size settings of the Back Wall and Green Area next to the virtual graphics.
• Square Corners: Shows the corners of the green area. This is useful for visualizing how the real right-angled corners would appear if there were no curvature in the green screen.
• Sizes and Square Corners: Displays both the size information and the square corners.

Custom Model - Green Area Mode

Custom Model Green Area Mode allows you to define any arbitrary green screen setup.

Green Model

To use this mode, you must provide a 3D model of your green screen and connect it to the Green Model pin of the Camera compound:

Important: The 3D model must not have any material applied to it.

To ensure no material is imported, right-click on the model’s compound and select Set Importer Options.... In the pop-up window, uncheck Import materials and click OK:

Margin Model

If you plan to use 3D Clean Plate or Window Mask Mode, you will also need to connect a model to the Margin Model pin of the Camera compound (located directly below the Green Model pin).

If you are using Window Mask Mode, we strongly recommend using a different model for the Margin Model pin than the one connected to the Green Model pin. This model should have the same layout, but be slightly larger in all directions. It should act similarly to the Margin Size parameter in other Green Area Modes, meaning it should overlap the edges of your physical green screen. While the Green Model should never overlap the edges.

Model Transformation

Use the Model Transformation parameter to position the model where it is located in your real world.

Put Model In Front of Camera

After pressing Put In Front, the model will move in front of the current camera view. This action automatically overwrites the Model Transformation parameter's position.

Monitor Mode

Functions the same way as the Curved Green Area Mode's Monitor Mode.

Free Camera

This option detaches the camera from your tracked real-world camera and allows you to navigate freely within the virtual studio. It helps you inspect your imported model to verify that everything is modeled correctly and, for example, that all vertices are present.

Show Floor

This option shows or hides a virtual floor at zero height. It helps you correctly place the 3D model by allowing you to check the exact heights and positions of the model’s parts.

Floor Size

Specifies the dimensions of the virtual floor. You can adjust this setting only if Show Floor is turned on.

Cube - Green Area Mode

Base Cam Transform

The Base Cam Transform parameters allow for aligning the orientation of all virtual Walls with our real-world physical studio.

If the tracked camera and the green screen are on the same floor (height), then do not adjust the Y parameter (which is the second box of the Pos row). Otherwise, it is suggested to use the CTRL + SHIFT key while editing the Y value for fine adjustments.

The Base Cam Transform parameters are as follows:

• Pos: Position, from left to right, these are the X, Y, and Z coordinates of the virtual Walls. These parameters use meters for measurement.
• Rot: Rotation, from left to right, these are the rX, rY, and rZ coordinates of the virtual Walls. These parameters use degrees for the measurement.

NOTE: You can learn more about Camera and Head Transformations here.

Walls

The Base Cam Transform is best used in conjunction with the Wall parameters. They allow for the independent positioning of the virtual Walls. These parameters use meters as the unit of measurement and are named according to the camera’s perspective.

• Front Wall: The Wall in front of the camera, usually the green screen itself.
• Left Wall: The Wall to the left of the camera.
• Right Wall: The Wall to the right of the camera.
• Back Wall: The Wall behind the camera. In a bigger green screen studio, the camera may be able to move more than 180 degrees within the green screen.
• Ceiling: The ceiling of the real-world physical studio.
• Floor: If the tracked camera and the green screen are on the same floor (height), then do not adjust this parameter.

Greens

Use the Green parameters to crop out the unnecessary parts of the video input. The Green parameters use meters for measuring, just like the Wall parameters.

• Green Front Left: adjusts the crop mask's left edge on the Front Wall.
• Green Front Right: adjusts the crop mask's right edge on the Front Wall.
• Green Left: adjusts the crop mask's left side on the Left Wall. Useful when the green screen is a corner setup.
• Green Right: adjusts the crop mask's right side on the Right Wall. Useful when the green screen is a corner setup.
• Green Top: adjusts the crop mask's height on the Front Wall.
• Green Floor: adjusts the crop mask's length on the Floor (the higher the value, the closer its edge will reach towards the camera).
  

Margin Size

Same as the Curved Green Area Mode's Margin Size.

Show Floor Zero

This parameter highlights the zero point of the floor. A 1 by 1 meter area is highlighted around the studio's 0x0 coordinates.

Mask Edge Fade

The Mask Edge Fade parameter softens the edges of the cropped-out video. This can help prevent harsh edges when the keying is imperfect.

For instance, if you plan to retain shadows during the keying process, the Mask Edge Fade can ensure that the shadow does not have a hard cut when it extends beyond the Studio's crop mask.
Below is an example illustrating a shadow encroaching on the studio's crop mask. By using the Mask Edge Fade parameter, the shadow is seamlessly blended in the last image:

Keyed	Studio	Final with 0 Mask Edge Fade	Final with 0.25 Mask Edge Fade

Walls Opacity

Lowering the Wall Opacity will make the virtual helper grid (Walls and Green Area) more transparent, allowing more of the video input to be visible. Conversely, increasing the Wall Opacity will make the helper grid more opaque, thereby making the video input less visible.

Show Green

The Show Green parameter shows or hides the Green area. When disabled, it allows you to clearly see the results of the crop mask, highlighting the part of the video input that you want to use.

Show Keyed

The Show Keyed parameter displays the keyed input behind the Green grids. When enabled, it helps you better define the edge of the green (crop mask). It is suggested to use this setting with the above Show Green parameter turned off.
For example, if your green screen has some wrinkles near the edge that the keyer cannot handle, these imperfections will appear as black over a white background, allowing you to easily adjust the green area to exclude them.

Show Non-Green

The Show Non-Green parameter shows or hides the non-green Walls. When disabled, it allows you to clearly see everything that has been cropped out of the video input.

Show Margin

Displays the margin as black virtual graphics. The margin is determined by the Margin Size parameter, or by the Margin Model when using the Custom Model Green Area Mode.

Show Garbage Mask

Shows or hides the Garbage Mask’s virtual graphics in the Studio view. More information about Garbage Masks can be found below.

Garbage Mask

You can specify extra non-green areas by connecting 3D models to the Garbage Mask pin.
These models, for example, can mask out a studio lamp in front of a green or other studio equipment that may go over the edges of your green screen from certain angles of your studio camera.

Here, a studio lamp is masked out from the green. The green area was set in the Studio control panel, and the mask was created by connecting Box body modules to the Garbage Mask pin.
NOTE: The example uses the basic primitive shape Box module of the Flow Editor. If you want to use your own models and you are not sure how to import models, then follow this document: Importing a 3D model

Without Garbage Mask	With Garbage Masks

NOTE: Show Green and Show Non-Green pins also affect these models' visibility and can help greatly when placing these models.

Show Holdout Mask

Shows or hides the Holdout Mask’s virtual graphics in the Studio view. More information about Holdout Masks can be found below.

Holdout Mask

Use HoldoutMask

Holdout Mask Edge Fade

HoldoutMask Example

For example, consider a chair that has a sticker with "Chair" written on it:

Because the sticker is green, it gets keyed out, causing the keyer to remove part of the chair. Using a Holdout Mask over the sticker, the keyer will ignore the area defined by the Holdout Mask's model, and the chair remains whole:

Without Holdout Mask	With Holdout Mask

Window Edge Fade

Similar to Mask Edge Fade, this parameter creates a smoothing between the keyed and non-keyed parts of the image. It also smooths out the Despill of the keying.

Additional Parameters

Show Origin Zero

This parameter highlights the zero point of the tracking system. A 1 by 1 meter area is highlighted around the tracking system's 0x0 coordinates:

Show Cameras

This parameter displays the other INPUTs' camera in the studio.
Only the cameras with their INPUT turned on are shown.

For example, if you are in the studio monitor of INPUT 1 and INPUT 2 is also turned on:

Then you can see the location of INPUT 2's camera in the studio view. If INPUT 2 happens to be the selected camera, it will be highlighted by red recording indicators:

Show Cam Heights

This parameter displays the height of each camera with a green-blue column and shows the exact height from the floor in meters above each camera.
It also highlights the camera's x and y positions on the floor with white lines. This can be helpful when comparing the positions of different cameras, especially when setting up and comparing two different tracking solutions.

NOTE: The green and blue colors of the column change after every 1 meter of height.

Show Cam Frustums

This parameter displays the camera's frustum from the camera's sensor. This can help visualize Zoom data.

NOTE: The higher the camera's focal length (measured in mm), the more zoomed in you are. Consequently, the camera's perspective will be narrower (FoV), and the frustum's area will be smaller.

Show Talent Trk

This parameter shows or hides the two helper graphics when using the Billboard panel's Talent Tracking system.

• The first helper graphic marks the exact position of the talent tracking system's tracker as a point. An accompanying text label displays the Billboard's ID. 
  
  You can read the Billboard IDs from here:
  
  
  
• The second helper graphic appears only if the Talent Tracking's Delta Talent Transf is changed from its default zero value. 
  This graphic shows the origin (zero point) of the talent tracking system, which helps you align its coordinate system.
  

Markers Visible

When the In Studio and Scene option is selected for the Markers Visible parameter, cameras are also displayed over the scene, a floor is shown starting from the zero position, and additional Markers can be defined with the Scene Marker Amount parameter.
With the In Studio and Scene option, two new parameters appear:

• Scene Floor Size: This parameter sets the size of the floor over the scene.
• Scene Marker Amount: This parameter allows you to add Markers over the scene.
  

Studio Marker Amount and Scene Marker Amount

Changing the Marker Amount from 0 reveals additional parameters for configuring markers in either the Studio or Scene. The number you select determines the total markers you can configure.
Each marker starts with the Marker Type parameter, allowing you to choose from eight different types. These markers can serve various purposes, for example, leaving notes in your studio or scene environment.

Continue to the Next Step