Back to Scene Control Panel
Introduction
In the following articles, we will describe how to use Billboards with Tracked Camera compounds in Aximmetry.
This article discusses the placement of billboards using Tracked Camera compounds in Aximmetry.
We assume that you have already set up your tracking system properly. If not, please check the How to Set Up Tracking Systems in Aximmetry article.
The Billboard is necessary for applying shadows, reflections, and occlusion of the talent in your virtual scene.
Before we continue, it is important to clarify a key point: when placing a billboard, we are not inserting the camera's input itself into the scene, as is done with a virtual camera (not tracked) compound's billboard. Instead, we are placing a surface onto which the camera’s input is projected from the camera’s perspective.
This approach is necessary because, for accurate realism, we need to maintain the correct relationship between the camera’s perspective and the video input, just as it is in the real world. This is one of the primary advantages of camera tracking: the video input is not arbitrarily placed within the virtual environment, but is instead always projected with precise perspective alignment between the tracked camera and the video input.
NOTE: If you wish to break this connection between the tracked camera’s perspective and the billboard, you can use Virtual Camera Movements of the CAMERAS control board.
We will start by describing a non-tracked Billboard setup so that later it will be easier to understand how tracked Billboards (Auto Position) work.
NOTE: To learn about setting up Shadows & Lights, please refer to: Tracked Camera Billboards: Shadows and Lights
NOTE: To learn about setting up Reflections, please refer to: Tracked Camera Billboards: Reflections
NOTE: To learn about setting up Occlusion, please refer to: Tracked Camera Billboards: Occlusion
Tracked Camera Compounds
After adding a tracked cam compound to your project, you will find the INPUTS control board on the menu bar.
All settings we are going to discuss in this article are on the INPUTS control board.
Use Billboards
Tracked Camera compounds by default don't use Billboards. To use Billboards, you have to turn the Use Billboards switch ON.
It will make all the difference after setting up reflections and shadows.
Here, you can see a scene with the Use Billboard turned OFF, so no reflections or shadows of the talent are visible.
Here, you can see a scene with Use Billboard turned ON. Reflections and shadows are now visible.
NOTE: These screenshots were created using the following stock studio scene: [Studio]:News Room\News Room - TrackedCam_3-Cam.xcomp
To enable Billboards in the scene, start by selecting the SCENE panel. Then, in the Pin Values panel, turn ON the Use Billboards option:
At this point, the camera’s image might disappear because the Billboard is placed at the scene origin (as this is the default transformation value of the Billboard). This is normal:
Placement of Billboard
It is recommended to hold the camera still during the placement of Billboards, and to place an object or a dummy talent where your actors will be during production.
There are several Billboard panels for segmentations and different camera inputs. We will start with the simple case of Camera Input 1 and no billboard segmentation.
Visibility
In order to properly position the billboard to suit our setup, you have to enable the Billboard's visibility.
Turn ON the lightbulb icon on the BILLBOARD 1A panel.
This is what the Billboard looks like after turning on visibility:
In the Visibility helper graphics, the smallest rectangle represents 10 x 10 cm, while the larger rectangle represents 50 x 50 cm:
To continue with the placement of the Billboard, select the BILLBOARD 1A panel, then you will see several parameters related to positioning the Billboard:
Transform Mode
The Transformation parameter defines the actual location of the Billboard. However, to fully understand what this transformation represents, we first need to discuss the parameter below it, called Transform Mode.
IMPORTANT: If you want to move the Billboard within the scene, it is usually best to move the whole scene in relation to the Billboard and the camera by using the SCENE panel’s Base Cam Transformation.
Transform Mode has two options, each representing a different reference point (coordinate system). When switching between them, the Transformation parameter’s value is automatically converted to that coordinate system:
-
Studio Relative (Real World)
This mode uses the camera tracking coordinate system. Using this option helps you place the Billboard, as it should primarily be positioned relative to where your talent is located in the real world, as viewed from the real-world camera. -
Scene Relative (Virtual World)
This mode uses the virtual scene’s coordinate system. Billboards, as part of the virtual scene, are placed in this coordinate system, which is the Aximmetry coordinate system used throughout the Flow Editor.
If you want to place objects in Aximmetry near the Billboard's location, you will likely need to switch to this option to get the transformation in the same coordinate system as everything else. This allows you to copy, paste, and reuse values from the Transformation parameter elsewhere in Aximmetry.
Be careful: in this mode, if you move the SCENE panel’s Base Cam Transformation, the Billboard will remain at the same location and will not move to the new scene location.
In addition to the coordinate systems described above, be aware of the following additional possibilities for different coordinate systems:
If you are using a Talent Tracking device, it will likely have its own coordinate system. You will need to match it to the studio’s coordinate system using the Talent Tracking’s Delta Talent Transf parameter.
If you have multiple cameras with different tracking systems (and therefore different coordinate systems), you can match their coordinate systems using the ORIGIN panel’s Delta Cam Transf parameter.
Put To Studio
When the Put To Studio trigger is pressed, the Billboard is automatically placed at the center of the Studio's green screen area by overwriting the Transformation parameter.
It is best to start with this when setting up your Billboard's position.
Put In Front
When the Put In Front trigger is pressed, the Billboard is positioned directly in front of the current camera view by overwriting the Transformation parameter.
Transformation
The Transformation parameter allows you to manually place the Billboard.
The Pos (Position) parameters adjust the following axes from left to right: X (left–right), Y (up–down), and Z (near–far). The values are measured in meters.
The Rot (Rotation) parameters adjust the rotation around each corresponding axis (rX, rY, and rZ) with values measured in degrees.
The Scal (Scale) parameter adjusts the size of the Billboard along all axes.
Place the Billboard where your talent is located in the real world. You can even measure and use real-world distances while in Studio Relative Transform Mode.
If the Billboard is intersecting the floor and its height is set to zero (Pos Y = 0), it is recommended to use the Scene Panel's Detect Floor feature to accurately set the scene’s height. If the tracking system itself does not have the correct height, use the Scene panel’s Detect Origin feature instead.
We suggest placing the billboard a bit in front of the feet of the talent, considering their movement towards and away from the camera. See example:
NOTE: If you place the Billboard too far from the talent, the reflections are not going to align with the talent's feet.
Another option is to use the Transformation Gizmo and Edit Scene Settings to move the Billboard. Either way, adjust its position carefully.
After positioning the Billboard, you can make fine adjustments to ensure it fits properly within the virtual scene and does not intersect with any virtual 3D models.
If you want to move the Billboard within the scene to a different place, it is usually best to move the whole scene in relation to the Billboard and the camera by using the SCENE panel’s Base Cam Transformation.
NOTE: The transformation pin will be overwritten if the Lock To Camera is turned on.
Horizontal Section
The Horizontal Section of the billboard allows you to place the billboard at the correct depth without clipping the feet.
This feature also helps with positioning shadows and prevents unnatural clipping of 3D virtual objects and effects that are close to the ground, such as grass.
This feature is sometimes referred to as an L-Shape Billboard.
When you increase its value from zero and have the Visibility button enabled, you will see a new section of the billboard appear at the bottom. This section is placed horizontally, in contrast to the rest of the vertical billboard:
NOTE: This feature currently does not work correctly when virtual camera movement is also used.
The Case of Billboard Segments
Tracked Camera compounds by default can handle up to 4 Billboards per video input. These Billboards cannot duplicate the image of their respective video input. They merely allow for cropping different parts of the same full-screen video input.
The Billboards are distinguished by a number and a letter, e.g., BILLBOARD 1A.
The number '1' indicates that this Billboard belongs to the first Camera's input (INPUT 1). The letter 'A' indicates that this is the first Billboard segment of this video input.
For INPUT 1, you'll find the following Billboards: BILLBOARD 1A, BILLBOARD 1B, BILLBOARD 1C, and BILLBOARD 1D
NOTE: The process of placing subsequent Billboards of the same video input is the same.
Placement of Billboard Multiple Billboard Segments
The use of BILLBOARD 1B, 1C, and 1D is optional. Therefore, you don't necessarily have to turn them on and use them.
They are only needed when you, e.g., have multiple talents in the full-screen video input, and you want to separately crop different parts of the video input to display the talents on multiple Billboards.
For example, this can also be helpful when using multiple Talent Tracking devices, or when you want virtual objects to appear in front of one talent (Billboard) and behind another.
BILLBOARD 1B, 1C, and 1D are - by default - turned off.
BILLBOARD 1B, 1C, and 1D can be placed in exactly the same way as BILLBOARD 1A. Simply select their respective panels and continue as it was discussed in the Placement of Billboard section.
Placement of Billboard of Other Cameras
The process for placing BILLBOARD 2A and BILLBOARD 3A, or any other tracked camera BILLBOARD, is identical to that for BILLBOARD 1A. For simplicity, we will use BILLBOARD 2A as an example.
Switch Cameras
Head to the CAMERAS control board, then to the SELECT CAMERA panel to select CAM2 for placing BILLBOARD 2A.
Then, head back to the INPUTS control board.
Align Tracking Coordinate System
If this camera uses a different coordinate system than the first one (such as if it uses a completely different tracking system) and there is no other way to align them, you can align the coordinate systems within the camera compound.
This is best performed in the Studio view of the MONITOR panel, where you can see each enabled camera rendered in the virtual studio space at its tracked position.
To properly align the different cameras within your camera compound, use the ORIGIN panel’s Delta Cam Transf or Detect Origin feature. For more information, refer to the ORIGIN panel's documentation.
NOTE: The tracking coordinate systems of different camera inputs do not necessarily need to be aligned. You can place camera 2’s origin point at any position that suits your production requirements. Typically, precise alignment is only required if the cameras are seeing the same talent(s) or studio environment.
Place the Billboard
After the alignment is complete, place BILLBOARD 2A using the parameters that you learned about in the Placement of Billboard section.
Lock To Camera
This is a legacy feature of the tracked green camera compounds, which was primarily used to place the Billboard before the introduction of the Transform Mode feature.
When enabled, it locks the Billboard to the camera’s movement at all times. The Billboard will remain parallel to, and maintain a fixed distance from, the tracked camera’s plane as the camera moves.
NOTE: Turn this feature on when you want the Billboard to follow the tracked camera's movement.
NOTE: Lock to Camera overwrites the previously set Transformation values of the Billboard (see: Transformation).
When Lock To Camera is set to On, additional parameters appear:
These parameters are discussed below:
Locked Distance
This value sets the distance between the Billboard and the tracked camera in meters.
When setting this value, please consider the area the Talent is going to cover when moving around during production. If the Talent walks closer to the camera than the distance specified here, the feet of the Talent will be cut off by the bottom of the Billboard. As the talent walks back, the shadows and reflections will become less accurate.
Locked Width
This value sets the width of the Billboard in meters.
When setting this value, please consider how far the talent is going to move to the left and right (parallel to the camera). If the talent walks 'off' the billboard, the image will be cut off by the edge of the Billboard.
Locked Height
This value sets the height of the Billboard in meters.
NOTE: It is recommended to keep a safe margin between the top border of the Billboard and the talent's head to avoid accidentally cropping it during production, if the talent steps towards the camera.
Locked Offset X & Y
This value sets the offset of the center of the Billboard from the center of the video input in meters.
Locked Offset X sets a horizontal offset.
Locked Offest Y sets a vertical offset.
Locked Rotation
This value sets the rotation of the Billboard around its center in degrees.
This feature is useful in cases where, e.g., your Talent is going to move in a certain direction during the show. You can set the Billboard parallel to that direction, keeping the reflection and shadows more accurate.
Billboard Tracking (Auto Position)
If the talent will walk on the green screen during production, we recommend using an Auto Position feature to track the talent’s location. This allows the position of the virtual billboard to match the talent’s movements. As a result, all reflections and shadows are updated in real time, even if the talent moves toward or away from the camera.
There are two methods for automatically tracking the talent’s location and synchronizing the billboard’s position within the virtual environment: Talent Tracking and Optical Tracking. We recommend using Talent Tracking, as Optical Tracking has more limitations compared to Talent Tracking.
Talent Tracking
Talent Tracking operates by having a tracking system, usually an Object Tracking System, attached to the talent in the real world.
To begin setting it up, set the Auto Position pin value to Talent Tracking:
Tracking Device
And under the Tracking Device, select the tracking system you are using for the talent.
Tracking Delay
If you discover that the tracking is either late or early (there is a delay), you can adjust the delay at the Tracking Delay pin.
Delta Talent Transf
This parameter allows you to align the coordinate system of the Talent Tracking device with that of the camera. Use this if the talent tracking and camera tracking systems differ, and you cannot align their coordinate systems by any other means.
Put In Front
Clicking Put In Front's Trigger will move the talent’s tracked position directly in front of the current camera view. This action automatically overwrites the position values in the above Delta Talent Transf parameter.
Optical Tracking
This method tracks the position of the talent directly on the camera screen and attempts to match the Billboard’s movement accordingly. However, it is less reliable than using an object tracking system.
We recommend that the talent not move too far from their original position and that any movement is kept slow and controlled.
There are three options to choose from:
NOTE: In non-Unreal camera compounds, only two Optical Tracking options are available, and both are “Depth Only” modes (even if “Depth Only” is not shown in the option name).
- Optical Depth Only (Per Input): Moves the billboard only in depth (closer to or farther from the camera). This option does not support Multiple Billboards (Per Input), so if you are using the B, C, or D billboards, do not choose this option.
- Optical Depth Only (Per Billboard): Moves the billboard only in depth (closer to or farther from the camera). But also applies positional cropping when using Multiple Billboards (Per Billboard), meaning the optical detection is limited to the selected billboard’s area on the input. If you are using the B, C, or D billboards, choose this option instead of the other Optical Depth Only option.
- Optical (Per Billboard): Moves the billboard not only in depth, but also in any horizontal direction. This option also applies positional cropping when using Multiple Billboards (Per Billboard), meaning the optical detection is limited to the selected billboard’s area on the input.
Auto Pos Offset
This parameter sets an offset for the Billboard’s optical tracking in depth. Adjust only if necessary.
