Tracked Camera Inputs

Introduction

This document discusses how to set up a tracked camera input using LEDWallCam compounds.

For all LEDWallCam compounds, you will find the INPUT # panel on the INPUTS control board after importing the camera compound.

In the example below, we imported the following compound: 
[Common_Studio]:Camera\LEDWallCam\LEDWallCam_3-Cam_4-Wall.xcomp

INPUT # Control Panel

You can turn each input On or Off with the power button icon on the panel. This will save your PC's resources, for example, if you have eight INPUT panels, but for production, you only need six, then you can turn off the other two.

Setting Up a Tracked Camera

On the INPUTS control board, if the INPUT # control panel is selected, you can see the input options on the right side of the screen in the Pin Values tab.
NOTE: LED Wall Production requires live video inputs.

The pins and their functionalities are described below, with this segment first discussing the most frequently used pins, while the remaining pins are discussed in the Additional Parameters segment.

Camera Devices

In the case of TrackedCam Compounds, the most important input devices on the INPUT # panel are the Camera Device and the Tracking Device.

By default, Mapped #1 is the default input for both of them, but this will only work if you have mapped any input to the #1 ID.

NOTE: More on mapping devices here.
NOTE: More on supported inputs here:

• Capture Cards
• Tracking Systems

Camera Mode

Select AUTO for Aximmetry to automatically set the input resolution and framerate. You can specifically set an arbitrary resolution and frame rate if you need to.

NOTE: With older capture card models, the AUTO feature may not work. If you are using AUTO mode and cannot see any video input, try manually specifying the correct input resolution and framerate.

Tracking Device

The correct order of setting up a tracking device is as follows, once you have already finished setting up your Camera Device:

• Define the Tracking Device
• Define the Calibration Profile in Aximmetry - if needed - (more on this at Setting Up Tracking Device)
• Check and set the Tracking Delay for the Tracking Device (more on this at Adjusting the Delays)

Calibration Profile

If you've used Aximmetry Basic Calibrator or Aximmetry Camera Calibrator to calibrate your camera lens and or tracking device, then select the calibration profile you've created here.

If your Professional Camera Tracking or Object Tracking system offers calibration solutions and you choose to use their solutions, then you do not have to specify the Calibration Profile parameter.

Zoom Device

Zoom Device is a secondary tracking device that provides additional tracking data in combination with the Tracking Device. There are three use cases for this parameter:

• specifying third-party Zoom encoder devices to receive additional lens data,
• specifying PTZ cameras' tracking data stream when PTZ cameras are tracked with additional tracking devices to determine their position,
• specifying the same tracking device that is specified at the Tracking Device parameter to be able to independently adjust the Tracking Delay and Zoom Delay parameters for it.

Delays

You can specify delay values for various inputs. Values are meant in frames.

• AR Tracking Delay

  Sets a delay value for the tracking data.

• AR Zoom Delay

  Sets a delay value only for the lens data.
  NOTE: Refer to the following point in the document to learn more about its use case scenarios: Zoom Device

AR Timecode Sync

Timecodes can have two sources:

• The studio devices can internally generate it (typically, camera bodies and professional camera tracking systems can do this).
• A timecode generator can provide it (timecode generators are external hardware, just like the genlock devices).

Using Timecode Generated by Studio Devices

When the studio devices generate their own timecode, these cannot be matched to each other. Therefore, to sync them, you'll need to first add a genlock device to genlock them, then you'll need to set the AR Tracking Delay.
NOTE: In rare cases, when the tracking and video originate from the same device, the timecodes for tracking and video can match without a timecode generator, as the device can manage it internally, like in the case of Aximmetry Eye.

Using External Timecode Generators

Timecode generators can send timecodes for each device, usually alongside a genlock/ref (short for reference) signal. This means that each device will use the same timecode; therefore, the video and tracking inputs can be synced using their shared timecode.
Typically, this also means you do not need to set the AR Tracking Delay.

By default, Timecode Sync is turned on. If you receive a "Cannot match timecode" error, your device(s) are likely generating their own timecodes, which will not match; therefore, they cannot be synced by them. In this case, turn off AR Timecode Sync.

AR Zoom Timecode Sync

The AR Zoom Timecode Sync syncs the AR Zoom Device according to the timecode, like the above AR Timecode Sync syncs the AR Tracking Delay.

Detect AR Tracking and AR Zoom Delay

From version 2023.2, Aximmetry offers a new feature: the ability to automatically detect the correct tracking (Detect AR Tracking Delay) and zoom delay (Detect AR Zoom Delay) values.

You can find these features among the INPUT # control panel's Pin Values:

To use it, you will need to point your camera at one of the LED Walls.
NOTE: It is recommended to use a higher shutter speed to avoid motion blur.
NOTE: It is recommended to place the camera at a reasonable distance from the LED Wall (avoid placing it too near, i.e., less than a meter away).
NOTE: It is recommended to place the camera in such a way that only one target is visible in the camera's field of view.

Switch On the detector you want to use. (Detect AR Tracking Delay, or Detect AR Zoom Delay) 
A message will appear.

After the target is detected by Aximmetry, a new message appears depending on if you are making AR Tracking Delay detection or AR Zoom Delay detection.
 

When it detects the delay value the following message appears and the value of the delay will be set automatically for the corresponding pin.

Additional Parameters

Tracking Forward

You can share tracking data input between multiple computers by forwarding it to one or more computers (e.g., in the case of a Multi-Machine setup).

Turn the Tracking Forward parameter On to forward tracking data from your computer to another. 

Tracking Fwd Port

To receive the tracking data forwarded from one Aximmetry computer to another Aximmetry computer, you need

• both/all computers to be connected to the same local area network,
• and a UDP port number specified by you in this parameter.

The 'sender' will broadcast the tracking data on the specified UDP port, making it available to every 'listener' connected to the local area network.

NOTE: To be able to receive and process the tracking data, it is recommended to use either the same camera compound as the one used to forward the tracking data or a camera compound that supports the use of tracking data inputs.

Timecode Master

Turn this On if you would like to set Aximmetry's timecode master to this input.

In Color Space

Aximmetry will automatically detect the color space, but if there is a specific color space you would like to use, you can select it here.

In Gamma Curve

Same as Color Space, but for the Gamma Curve.

Chroma Offset

Chroma Offset is used to realign the chroma (color) channels with the luma (brightness) channel to ensure that the pixels of the different channels are correctly positioned. Chroma channels (U and V or Cr and Cb) can have different formatting compared to the luma channel; this difference can cause the color and brightness to be very slightly offset in the image. Adjusting the Chroma Offset helps to rectify this issue.

No Input Image

The default no-input image is this:

You can browse your PC for a different image with this option.

Override Lens Parameters

• Override Lens Data

  By enabling this parameter, you can manually override the lens data received from your tracking device or lens encoders. The manually overwritten data will then be processed according to the lens Calibration Profile, if it is specified in Aximmetry.
  NOTE: The manual lens data is input through the Override Lens Data X input pins present on the LEDWallCam camera compound.

• Raw Override

  By adjusting this parameter, you can manually override the lens data received from your tracking device or lens encoders. The manually overwritten data will then be used as raw input, ignoring the Calibration Profile, if it is specified in Aximmetry.

• Override Zoom

By enabling this parameter, you can manually override the Zoom value of the incoming lens data by adjusting the value of the Zoom pin under the Override Zoom pin.

Override Lens Data Pin

There is one Override Lens Data pin for each INPUT on LEDWallCam compounds:

The Override Lens Data pin expects a vector containing 9 floating-point values:

1. 1. DescType - Description Type, where:
      
      • 0 = OpenCV (the default Aximmetry format; also used by Aximmetry Camera Calibrator)
        
      • 1 = Stype
      • 2 = Mo-Sys
      • 3 = TrackMen
      • 4 = Camix
   2. FOVX - horizontal Field Of View in degrees
   3. Aspect - image Aspect Ratio
   4. CX - Center Shift X component
   5. CY - Center Shift Y component
   6. K1 - Radial Distortion coefficient 1
   7. K2 - Radial Distortion coefficient 2
   8. K3 - Radial Distortion coefficient 3
   9. SensorWidth - the width of the used area of the camera sensor

NOTE: Although the vector has a unified layout, the exact interpretation of the data elements can differ based on the specified description type.
NOTE: An example of how to parse lens data and create such a vector is the [Common_Studio]:Compounds\Tools\Lens_Distort_Vanishing.xcomp compound.

External Lens Data

External Lens Data switches between Aximmetry reading the lens data from either the Calibration Profile or the tracking system (only when the tracking system provides lens data).

Covering all possibilities, the source of the lens data changes as follows when the Calibration Profile is specified:

Calibration Profile is specified	External Lens Data is On	External Lens Data is Off
The tracking system provides lens data AND no Zoom Device is specified	tracking system	calibration profile
The tracking system does not provide lens data OR a Zoom Device is specified	calibration profile	calibration profile
Playing back recorded tracking data	recorded data	calibration profile

When the Calibration Profile is not specified:

Calibration Profile is not specified	External Lens Data is On	External Lens Data is Off
The tracking system provides lens data AND no Zoom Device is specified	tracking system	fixed default value
The tracking system does not provide lens data OR a Zoom Device is specified	fixed default value	fixed default value
Playing back recorded tracking data	recorded data	fixed default value

The fixed default values are FOVX = 60, Aspect = 16:9, no distortion.

Tracking Dejitter

Tracking Dejitter can stabilize tracking when the camera is not moving. Many tracking systems have their tracking data change constantly, even when the tracker is stationary (due to their measuring precision). Usually, these changes are so small that they are not visible. However, when zooming in, these changes can become much more noticeable.

Enabling the Tracking Dejitter will lock the camera to a fixed position as long as the camera does not move more than the Dejitter Pos Thresh or rotate more than the Dejitter Rot Thresh.

You should turn on Tracking Dejitter only if absolutely necessary, as it can cause a slight lag in tracking at the moment when the camera starts to move. Additionally, check if the tracking system provides an alternative, as the manufacturer will know their system's measuring precision and may offer a better solution than Tracking Dejitter.

NOTE: Tracking Dejitter does not fix jittering while the camera is moving.

Dejitter Pos Thresh

Dejitter Pos Thresh specifies the threshold value for position data. Data fluctuation lower than the threshold value will be ignored/discarded by Aximmetry.
NOTE: For this parameter to affect the processing of the tracking data input, Tracking Dejitter must be turned on.

Dejitter Rot Thresh

Dejitter Rot Thresh specifies the threshold value for rotation data. Data fluctuation lower than the threshold value will be ignored/discarded by Aximmetry.
NOTE: For this parameter to affect the processing of the tracking data input, Tracking Dejitter must be turned on.

Frustum Outline Color

NOTE: To learn about how to turn Frustum Outline on, please refer to the following document:
Show Cam Frustum

Click on the colored rectangle to pick any color for the Frustum Outline.