Search
Start typing to search...

Recording camera tracking data

Author:

Introduction

We differentiate between two kinds of tracking data recording serving different purposes. They can be used independently of each other in any combination.

Input recording

Its purpose is to enable offline rendering of the final show using pre-recorded camera images  and tracking data.

Video recorded: the raw, unmodified camera input image.

Tracking data recorded: the raw, unmodified data coming from the camera tracking system and also the full lens distortion data either coming from the tracking system or calculated from Aximmetry's own calibration file. They are stored in an Aximmetry-specific file format (xdata).

All the inputs or a selection of them can be recorded individually.

Final composite recording

Its purpose is to enable further 3D post-production work with a third-party rendering software using the final composite output of Aximmetry, both image and tracking data.

Video recorded: the final composite of the virtual background and the keyed camera images.

Tracking data recorded: the final camera position and focal length data which are already modulated by the Base Cam Transform and ORIGIN settings and the VR path motions. It's stored in an FBX file in order to be able to use from any third-party system. Please note that currently the the lens distortion data cannot be recorded with this method. Important that this feature can be used with VirtualCam* compounds as well to record virtual camera motions.

Input recording

This functionality is added to the INPUTS control board of each TrackedCam*.xcomp.

RecordingTrackingData Image1.png

Timecode

The pairing of the camera video frames with the corresponding tracking data frames is based on timecode. Therefore it is mandatory to provide the system with a running time code which is compatible with the current system frame rate.

Using individual timecode

By default each recording has its own timecode that starts from zero when the recording begins. This is the simplest way to provide a synchronization

Using input timecode

However you may want to use a timecode provided by the camera itself via SDI. It can be either the running internal timecode of the camera or can be taken from a central timecode generator attached to the camera.

This way you even have the freedom to record the image on the camera itself for the best possible image quality, then you'll still be able to pair it with the tracking data recorded in Aximmetry.

For this turn on the Use Input TC option on the RECORD panel:

RecordingTrackingData Image2.png    RecordingTrackingData Image3.png

Recording

Select the RECORD panel.

RecordingTrackingData Image4.png

Select which inputs you want to record. Please take into consideration that recording too many inputs simultaneously may impact the system performance.

RecordingTrackingData Image5.png

You can sepecify the usual parameters of the recording format. For all cameras the same settings will be used, but the file names will be suffixed individually based on the tags you provide (Output Suffix X). Furthermore all takes will have an individual index number. Since these recordings will be re-used later we recommend using a higher bitrate (also depending on your storage capacity).

RecordingTrackingData Image6.png

Press the red circle button to start the recording.

RecordingTrackingData Image7.png

On the preview screen you'll see a red counter displaying the recording time. Also the indices of the currently recording inputs are indicated.

RecordingTrackingData Image8.png

Use the white stop button to stop the recording. On stopping the last recorded file(s) will be automatically entered into the Input X properties of the PLAYBACK panel thus enabling the immediate checking of the recordings.

RecordingTrackingData Image9.png

Regarding the playback see later in this documentation.

Please note that beside the video file an xdata file with the same name containing the tracking data is also created.

Recording the image on the camera

As we discussed above you might want to record the image on the camera itself for the highest possible quality and only record the tracking data with Aximmetry. For that choose the Tracking Only option:

RecordingTrackingData Image10.png

Make sure your camera sends a running timecode through SDI and also uses the same timecode for the recording. Set Aximmetry to use this external timecode.

RecordingTrackingData Image11.png

Please note that in Tracking Only mode you still get a recorded video, but it's created with highly degraded quality for the minimal performance impact. It only serves as a placeholder. Later when you want to playback the material replace this placeholder by copying the recorded file on your camera's SD card over it.

Audio

All inputs are automatically recorded along with its audio. By default the audio is taken from the same SDI/HDMI input the video is coming from. In order to use a separate sound card or any USB device for receiving audio turn on the Use Audio Device option for the given input and select the corresponding system audio device:

RecordingTrackingData Image12.png    RecordingTrackingData Image13.png

To hear the incoming audio turn on the speaker button on the AUDIO panel and in the Monitor Device property specify which Windows audio output device you want hear the audio on (usually the default Primary Driver will be fine):

RecordingTrackingData Image14.png  RecordingTrackingData Image15.png

In order to synchronize the incoming video and audio signals use the Audio Dev Delay property. It is expressed in seconds.

RecordingTrackingData Image16.png  RecordingTrackingData Image17.png

Here you can also adjust the Audio Level. It's relative and expressed in decibels, 0 means no change.

Playback

In order to playback a recording select the video file(s) on the PLAYBACK panel. You can specify a file for each input separately. If one of the Input X properties left blank the corresponding input will continue to receive the live signal even in playback mode.

RecordingTrackingData Image18.png    RecordingTrackingData Image19.png

As we mentioned earlier, after each recording session the last recorded file(s) are automatically entered to these properties.

Please note that here you select a video file, but beside it a corresponding xdata file with the same name must exist as well.

Turn on the Play button on the panel to put the system into playback mode:

RecordingTrackingData Image20.png

The preview screen will indicate the playback mode and also the indices of the inputs that actually involves in the playback (the other inputs continue to receive their live signal).

RecordingTrackingData Image21.png

By default you will only see the the first frame of the recording freezed. The reason for this is that the system expect you to use a Sequencer to play back the video and tracking, because this way it is very easy to add any synchronized animation to your show via additional sequence tracks. So please add a Sequencer and connect it this way:

RecordingTrackingData Image22.png

If you use recordings for multiple inputs connect the Sequencer for all of them:

RecordingTrackingData Image23.png

If you open the Sequence Editor by double-clickink the Sequencer you will see a track for each video file you specified on the PLAYBACK panel.

RecordingTrackingData Image24.png

Now you can Play/Stop or seek the sequencer as usual to see the recorded material along with the tracking.

RecordingTrackingData Image25.png

As we mentioned you can add any additional animation tracks that will play in sync with the recorded material.

RecordingTrackingData Image26.png

RecordingTrackingData Image27.png

On using the Sequencer in general please see this documentation.

Combining playback mode with recording

You can leave the playback mode on even while you are recording.

Firstly this way you can instantly see the results when you stop recording, since the last recorded file is automatically selected for playback.

Secondly the files selected for playback on the inputs you didn't select for recording still remain in playback. To be more clear let's suppose you have a recording for all the inputs:

RecordingTrackingData Image28.png

Now you start a new recording with the following settings:

RecordingTrackingData Image29.png

In this case the current live feed on Input 3 will be recorded while the previously recorded material for Input 1 and 2 will be played back simultaneously, and can be seen e.g. in MATRIX mode along with the live feed.

RecordingTrackingData Image30.png

You can also instruct the system to restart the Sequencer automatically each time you start or stop a recording by making the following connection. This helps both the above usage scenarios.

RecordingTrackingData Image31.png

Playback time display

If you make the following connection you will also get a display of the current playback postion.

RecordingTrackingData Image32.png

RecordingTrackingData Image33.png

Please note that the displayed time is the position of the Sequencer, not the video file itself. The two can differ, because you may shift the video track within the sequencer to better suit your animation timeline.

Tracking data

When you're recording an xdata file is also created. It contains the full tracking data, including the camera lens data. If you used an Aximmetry lens file created with Camera Calibrator then the xdata will contain the final calculated FOV, lens distortion etc. data for each frame. It means that when you play it back you'll be guaranteed to see exactly the same result you got when you did the recording.

However the raw zoom/focus encoder positions are recorded as well. It means you have the freedom to improve/adjust the lens calibration even after you recorded the material. In order to use an improved or different lens file during playback simply turn off External Lens Data and specify the modified lens file in Tracking Mode.

RecordingTrackingData Image34.png

You can also use the Manual Lens mode to override the recorded lens data.

Final composite recording

For this scenario the tracking data recording functionality is added to the Record_* compound you normally use to record the final image.

RecordingTrackingData Image35.png

If you receive audio from multiple inputs simultaneously then connect the Audio 2, 3 pins as well.

On using the Record compund in general please see this tutorial.

Do not forget the connect the Record Data pin, it carries the tracking and other information to be recorded.

By default the compound only records the video and audio, this is the most common usage. In order to record the tracking data as well, turn on the TRACKING panel:

RecordingTrackingData Image36.png

On the panel you can specify whether you want to generate the FBX file in ASCII or binary format. The default is the latter, it's more compact.

RecordingTrackingData Image37.png    RecordingTrackingData Image38.png

Timecode

The pairing of the final composite video frames with the corresponding tracking data frames is based on timecode. Therefore it is mandatory to provide the system with a running time code which is compatible with the current system frame rate.

The tracking data is recorded into an FBX file in the form of several camera animation curves. The curve key timestamps will correspond to the frame timecodes.

Using individual timecode

By default each recording has its own timecode that starts from zero when the recording begins. This is the simplest way to provide a synchronization

Using system Master Timecode

Aximmetry has its own internal timecode that starts from zero when you start Composer. You can use this timecode as well in order to differentiaite between different takes of recording. To use if turn on Use Master TC on the RECORDER panel:

RecordingTrackingData Image39.png    RecordingTrackingData Image40.png

Using input timecode

You may want to use a timecode provided by one of the cameras itself via SDI. It can be either the running internal timecode of the camera or can be taken from a central timecode generator attached to the camera.

The final composite may contain cuts between multiple camera inputs. But the source of the timecode have to be constant, so you have decide which camera input's timecode will be used. Then turn on Timecode Master for that single input:

RecordingTrackingData Image41.png    RecordingTrackingData Image42.png

This option means that the Aximmetry internal Master Timecode will be overwritten by the incoming timecode of the camera input, in other words the camera timecode becomes the Master Timecode in Aximmetry. By combining this with the Use Master TC option discussed previously you effectively get the camera timecode passed to the final recording.

Recording

Use the RECORDER panel's properties to setup the output video format as usual. Then use the two buttons of the panel to start and stop the recording.

RecordingTrackingData Image43.png

The time indicator will be displayed on the right side of the preview screen to differentiate from the input recording's one.

RecordingTrackingData Image44.png

Camera FOV and lens distortion

The resulting FBX file will contain a camera object with aperture properties and animated Focal Length that gives the same aspect ratio and field of view as the Aximmetry output image.

But please note that currently the the lens distortion data cannot be recorded this way.

Using with VirtualCam

It is worth to note that this recording functionality works together with any VirtualCam compound as well. In this case the FBX will contain the animated virtual camera positions.

RecordingTrackingData Image45.png

Combined usage

The input recording and the final composite recording functionality can be used independently of each other in any combination.

E.g. you can start a recording for both of them simultaneously.

But what is more useful you can playback a pre-recorded input while recording the final output along with the tracking data for further post-processing.

Article content

Close
1/10