Back to Tracked Camera Compounds
Introduction
In this document, we discuss how to set up external and internal input using TrackedCam compounds.
- External input is usually live input coming from a studio camera, but it can come from other sources as well (e.g.: tracking device, audio device).
- Internal input comes from inside Aximmetry after importing a source file, therefore no external hardware is required for it. These are usually pre-recorded videos and tracking data, though they can be audio recordings as well.
For all Tracked Camera compounds, you will find the INPUT panel on the INPUTS control board, after importing the compound.
In the below example, we imported the following compound:
[Common_Studio]:Camera\TrackedCam\TrackedCam_3-Cam.xcomp
INPUTS Control Board Content
After navigating to the INPUTS control board, you will see a lot of panels.
Each row of panels starting with an INPUT # panel corresponds to a single Input. Additional panels - without numbers in their names - correspond to all the inputs.
In this article, we will discuss the INPUT # panel.
If you are looking for other panel settings on the INPUTS control board, these articles will be helpful:
Or just simply follow the Tracked Camera Workflow category.
INPUT # Panel
You can turn each input On or Off with the power button icon on the panel. This will save your PC's resources, for example, if you have eight INPUT panels, but for the production you only need six, then you can turn off the other two.
External (Live) Input
External input can mean any input that is received from external hardware (e.g. camera, microphone, tracking device, another computer) preferably via cables (e.g. via SDI, SMPTE2110, NDI, etc.).
On the INPUTS control board, if the INPUT # panel is selected, you can see the input options on the right side of the screen in the Pin Values tab.
Camera Devices
In the case of TrackedCam Compounds, the most important input devices on the INPUT # panel are the Camera Device and the Tracking Device.
By default, Mapped #1 is the default input for both of them, but this will only work if you have mapped any input to the #1 ID.
More on mapping devices here.
More on supported inputs here:
Camera Mode
Select AUTO for Aximmetry to automatically learn the input resolution and framerate. You can specifically set an arbitrary resolution and frame rate if you need to.
NOTE: With older capture card models the AUTO feature may not work. If you are using AUTO mode, and cannot see any video input, then you can try manually specifying the input resolution and framerate.
Tracking Device
The correct order of setting up a tracking device is as follows once you have already finished setting up your Camera Device:
- Define the Tracking Device
- Define the Calibration Profile in Aximmetry - if needed - (more on this at Calibration Profile)
- Check and set the Tracking Delay for the Tracking Device (more on this at Delays)
Calibration Profile
If you've used Aximmetry Basic Calibrator or Aximmetry Camera Calibrator to calibrate your camera lens and or tracking device, then select the calibration profile file you've created here.
If your Professional Camera Tracking or Object Tracking system offers calibration solutions and you choose to use their solutions, then you do not have to specify the Calibration Profile parameter.
Zoom Device
Zoom Device is a secondary tracking device that provides additional tracking data in combination with the Tracking Device. There are three use cases for this parameter:
- specifying third-party zoom encoder devices to receive additional lens data,
- specifying PTZ cameras' tracking data stream when PTZ cameras are tracked with additional tracking devices to determine their position,
- specifying the same tracking device that is specified at the Tracking Device parameter to be able to independently adjust the Tracking Delay and Zoom Delay parameters for it.
Tracking Forward
You can share tracking data input between multiple computers by forwarding it to one or more computers (e.g. in the case of a Multi-Machine setup).
Turn the Tracking Forward parameter On to forward tracking data from your computer to another.
Tracking Fwd Port
To receive the tracking data forwarded from your computer to another, you need two pieces of information: your computer's IP address and the UDP port number specified by you at this parameter.
Use Audio Device
Turn the Audio Device parameter On to receive an audio input.
Audio Device
Select here your audio input device.
Timecode Master
Turn this ON if you would like to set Aximmetry's timecode master to this input.
In Color Space
Aximmetry will automatically detect the color space, but If there is a specific color space you would like to use, you can select it here.
In Gamma Curve
Same as Color Space, but for Gamma Curve.
Chroma Offset
Chroma Offset is used to realign the chroma (color) channels with the luma (brightness) channel to ensure that the pixels of the different channels are correctly positioned. Chroma channels (U and V or Cr and Cb) can have different formatting compared to the luma channel, this difference can cause the color and brightness to be very slightly offset in the image. Adjusting the Chroma Offset helps to rectify this issue.
No Input Image
The default no-input image is this:
You can browse your PC for a different image with this option.
Manual Lens Parameters
-
Manual Lens
By enabling this parameter, you can manually override the lens data received from your tracking device or lens encoders. -
Manual Zoom
By adjusting this parameter, you can manually change the field-of-view (FOV; focal length) values of your tracking data. (only when Manual Lens is enabled)
-
Manual Focus
By adjusting this parameter, you can manually modify the focus distance value of your tracking data (only when Manual Lens is enabled).
NOTE: For this adjustment to take effect, the render compound must support this feature and the virtual camera must have a shallow focus depth.
When the Manual Lens is turned ON, the source of lens data changes as follows:
| Manual Lens is On |
Manual Lens Distortion pin is connected | Manual Lens Distortion pin is not connected |
| FOV (Zoom) comes from | Manual Focus parameter |
Manual Focus parameter |
|
Focus comes from |
Manual Lens Distortion pin |
Manual Zoom parameter |
|
Lens Distortion comes form |
Manual Lens Distortion pin |
- |
Manual Lens Distortion pin
There is one Manual Lens Distortion pin for each INPUT on tracked camera compounds:
The Manual Lens Dist pin expects a vector containing 8 floating-point values:
-
- DescType - Description Type, where:
- 0 = OpenCV (the default Aximmetry format; also used by Aximmetry Camera Calibrator)
- 1 = Stype
- 2 = Mo-Sys
- 3 = TrackMen
- 4 = Camix
- 0 = OpenCV (the default Aximmetry format; also used by Aximmetry Camera Calibrator)
- FOVX - horizontal Field Of View in degrees
- Aspect - image Aspect Ratio
- CX - Center Shift X component
- CY - Center Shift Y component
- K1 - Radial Distortion coefficient 1
- K2 - Radial Distortion coefficient 2
- SensorWidth - the width of the used area of the camera sensor
- DescType - Description Type, where:
NOTE: Although the vector has a unified layout, the exact interpretation of the data elements can differ based on the specified description type.
NOTE: An example of how to parse lens data and create such a vector is the [Common_Studio]:Compounds\Tools\Lens_Distort_Vanishing.xcomp compound.
External Lens Data
External Lens Data switches between Aximmetry reading the lens data from either the Calibration Profile or the tracking system (only when the tracking system provides lens data).
Covering all possibilities, the source of the lens data changes as follows when the Calibration Profile is specified:
| Calibration Profile is specified | External Lens Data is On |
External Lens Data is Off |
| The tracking system provides lens data AND no Zoom Device is specified | tracking system |
calibration profile |
| The tracking system does not provide lens data OR a Zoom Device is specified | calibration profile |
calibration profile |
| Playing back recorded tracking data | recorded data |
calibration profile |
When the Calibration Profile is not specified:
| Calibration Profile is not specified | External Lens Data is On |
External Lens Data is Off |
| The tracking system provides lens data AND no Zoom Device is specified | tracking system |
fixed default value |
| The tracking system does not provide lens data OR a Zoom Device is specified | fixed default value | fixed default value |
| Playing back recorded tracking data | recorded data |
fixed default value |
The fixed default values are FOVX = 60, Aspect = 16:9, no distortion.
Delays
You can specify delay values for various inputs. Values are in frames.
-
Camera Delay
Sets a delay value for the video input. Do not adjust this parameter, unless it is required.
-
Tracking Delay
Sets a delay value for the tracking data.
-
Zoom Delay
Sets a delay value only for the lens data.
Please refer to this point in the document to learn more about its use case scenarios: Zoom Device -
Audio Dev Delay
Sets a delay value for the Audio Device's input.
Timecode Sync
Timecode generators can send timecodes for each device, usually alongside a genlock signal. This ensures that the video and tracking are synced. Typically, this also means you do not need to set Tracking Delay.
In rare cases, when the tracking and video originate from the same device, the timecodes for tracking and video can match without a timecode generator, as the device can manage it internally, like in the case of Aximmetry Eye.
By default, Timecode Sync is turned on. If you receive a "Cannot match timecode" error, your device(s) are likely sending timecodes, but they are not synced. In this case, turn off Timecode Sync.
Zoom Timecode Sync
Can sync the Zoom Device according to the timecode, like the above Timecode Sync syncs the Tracking Delay, Zoom Timecode Sync syncs the Zoom Delay.
Detect Tracking and Zoom Delay
From version 2023.2, Aximmetry offers a new feature: the ability to automatically detect the correct tracking (Detect Tracking Delay) and zoom delay (Detect Zoom Delay) values.
You can find these features among the INPUT# control panel's Pin Values.
To use it, you will need a red, blue, or green reference point in your studio. It could be any flat object or detail of even red, blue, or green color. In the example, we chose a red circle as our target.
NOTE: It is recommended to use a flat target for reliable results.
NOTE: It is recommended to choose a color different from our background.
NOTE: It is recommended to use a higher shutter speed to avoid motion blur.
NOTE: It is recommended to place the target at a reasonable distance from the camera (avoid placing it too near, i.e. less than a meter away).
You may use one of these 3 images as reference points:
(Click on one of the 3 circles for the full-resolution image).
You can show the image on your smartphone or on any other screen. Or print it out like in the example below.
Choose the color of the target in the Detect Target Color dropdown.
Switch On the detector you want to use. (Detect Tracking Delay, or Detect Zoom Delay)
A message will appear.
After the target is detected by Aximmetry a new message appears depending on if you are making Tracking Delay detection or Zoom Delay detection.
When it detects the delay value the following message appears and the value of the delay will be set automatically for the corresponding pin.
Tracking Dejitter
Tracking Dejitter can stabilize tracking when the camera is not moving. Many tracking systems have their tracking data change constantly, even when the tracker is stationary (due to their measuring precision). Usually, these changes are so small that they are not visible. However, when zooming in, these changes can become much more noticeable.
Enabling the Tracking Dejitter will lock the camera to a fixed position as long as the camera does not move more than the Dejitter Pos Thresh or rotate more than the Dejitter Rot Thresh.
You should turn on Tracking Dejitter only if absolutely necessary, as it can cause a slight lag in tracking at the moment when the camera starts to move. Additionally, check if the tracking system provides an alternative, as the manufacturer will know their system's measuring precision and may offer a better solution than Tracking Dejitter.
NOTE: Tracking Dejitter does not fix jittering while the camera is moving.
Audio Level
Changes the audio level in decibels (dB). A level of 0 means the audio's loudness is not modified. Quieter levels are measured as negative numbers, with -100 dB indicating mute. Any level above 0 dB can cause a loss in audio quality.
The INPUTs' Audio can be monitored by using the AUDIO panel:
Each INPUT's audio can be accessed using the Audio pins:
NOTE: There are various audio modules available in the Flow Editor that can process and mix different audio sources.
Internal (Pre-recorded) Input
Internal input means that instead of a live input from an external device (e.g. studio camera, tracking device, audio device), you use pre-recorded footage (e.g. video, tracking data, audio, image, image sequence, etc.) inside Aximmetry. This can be helpful for different reasons. For example, there is no need for the presence of your talent(s) for setting up the virtual scene, lights, etc.
Aximmetry is capable not only of using a wide range of pre-recorded footage and data but also of capturing them for various purposes (e.g. post-production).
You can learn more about pre-recorded footage and data here: How to Record Camera Tracking Data
Setting Up Internal Video and Tracking Input
In order to use a pre-recorded video input and tracking data with a TrackedCam compound, go to the PLAYBACK control panel on the INPUTS control board.
Select your pre-recorded video file on the Pin Values tab. Here you can add pre-recorded video inputs for every INPUT # control panel you have.
NOTE: You only have to add the video file. Aximmetry will automatically import the corresponding tracking data from the folder of the video file based on the video file's name (the video file and the tracking data file must share the same exact file name).
Once you are ready, press the Play button on the PLAYBACK control panel.
Navigate to the FLOW editor in the Menu bar.
With the right button of your mouse click on the Flow Editor's background, then select Insert Module... and add a Sequencer module.
Make the following connection with the TrackedCam compound:
Connect the Tracks pin of the Sequencer module to the TrackedCam compound's Playback Seq # pin. (In our case it is Playback Seq 1.)
Double-click on the Sequencer module to open the Sequence Editor panel. Use Space to start and pause the video.
Here you can learn more about the use of the Sequencer and Sequence Editor.
This will start playing the imported video and its tracking data on the Billboard.
Of course, the same exact steps should be done with the pre-recorded video as if it were a live input, for example, checking and adjusting delay values, keying, setting up the studio mask etc.
Continue to Keying
Once you have finished setting up your inputs, you can move on to the Keying:
