Introductions
Before we can talk about camera tracking, we have to discuss the kinds of cameras and the differences between them when talking about virtual productions.
Generally, we talk about two very different cameras:
- Physical Camera (also called Studio Camera),
- Virtual Camera.
The video feed of your talent is captured through the Physical Camera in your physical studio.
The virtual scene is shown through the perspective of the Virtual Camera. This is an idealized 'pinhole' camera.
Stationary/Fixed Camera
A Fixed Camera for virtual production simply means that there is no Physical Camera movement during the production. It is important to note that this applies not just to the position and rotation of the camera, but also to its zoom and focus. So there is no change in the angle, perspective, height, rotation, FOV, etc. of the video provided by the fixed Physical Camera during the production. However, you can achieve limited camera movements with virtual cameras.
Example of a stationary camera production:
Moving/Tracked Camera
A Tracked Camera for virtual production means that there is a tracking system reporting the change in the Physical Camera's position, FOV, angle, etc. during the production.
Example of a moving camera production:
NOTE: The use of a tracking system requires the installation of a tracking device onto the camera or its mount to follow its movements.
For the final production to be of the highest quality, the virtual scene's perspective needs to change in parallel with the physical world's perspective. In order to match the change in the 'view' of the Physical Camera to the view of the Virtual Camera, you need the exact transformation (position, rotation, and even zoom and focus data in some cases) of the Physical Camera, which is provided via a tracking system. This system tells the Virtual Camera how to show the virtual scene so that it matches the Physical Camera's movements.
Tracking systems come in many shapes and forms, each with its advantages and disadvantages.
Some tracking systems come with lens encoders that are used to keep track of the zoom, focus, and aperture values of your lens during production. You can use tracking systems without lens encoders but in this case, the zoom, focus, and aperture must not change during production.
When it comes to tracking, the most important part is to have a smooth result. One big factor is the precision of the tracking device, and the other is the synchronization capabilities.
Synchronizing the video input, tracking signal and the rendering of the virtual scene ensures that the virtual studio is moving together with the video feed from the studio camera.
If you would like to read more about tracking systems, please refer to this document: Tracking Systems.
