What is PTZ?
PTZ is an abbreviation for Pan-Tilt-Zoom. It is a mechatronics solution which can adjust Camera orientation, Zoom and Focus parameters over a software or controller.
What type of PTZ solutions are out there ?
There are two main kind of PTZ solutions
- “Motorized Pan/Tilt heads” + “Zoom Device”
- All in one “PTZ Camera”
Note: “Motorized pan/tilt heads” is not the same as the more advanced “Motion Control Rig” even if they can communicate with the same protocols.
Motorized Pan/Tilt head
Motorized Pan/Tilt heads is a Pan+Tilt system which can be used with multiple Compatible Cameras.
This is really useful if your production needs to:
- Use different lenses
- Existing camera which you already own and used to but in itself does not have tracking capabilities
- A very specific camera due to its properties
Lens Encoder (Zoom Device)
Lens Encoder (Zoom/Focus Device) is a separate device from the Pan/Tilt head, which can provide Zoom and Focus information. So If you use Motorized Pan/Tilt heads and you wish to have Zoom/Focus control. This provides the data to determine the Zoom/Focus values . It is either:
- Only a Zoom device like “Glassmark” (this provides fix focus only)
- There might exist cameras which use Direct DataConnection from the Camera to send accurate Zoom/Focus information.
You will additionally need parameters for lens distortion. Some Tracking systems can provide this information. While for others Aximmetry can measure it during the Calibration process.
Pan/Tilt head with built in Encoder
IMPORTANT: The above example does not have Tracking capabilities however it is possible there are similar system which might have.
PTZ Camera is usually a complete Pan-Tilt-Zoom-Focus system where the camera is built into the mechanism. PTZ cameras are easier to set up as they send Tracking, Zoom and Focus data together. PTZ cameras have been used by a big portion of our users because of their relatively easy operation and setup process.
"Motion Control Rig"
Motion Control Rig is actually a Robotic Camera System which provides Complete Tracking of the Camera, and they are not considered PTZ systems even though they can still use the same protocols as PTZ cameras. Or some of their sub variants only send the same type of information
Sending and Receiving Data to and from PTZ cameras.
Usually the user can send data to control the system, however only a handful of systems can provide feedback for accurate tracking.
Sending Control Data to PTZ system
Aximmetry Supports Receiving PTZ Data from multiple protocols, like
- Visca (obselate format, limited compatibility with new hardware. Please see your product description if it only receives PTZ controls through Visca or if it can also send back Tracking informations )
Over these protocols Aximmetry can provide accurate tracking for your production which you can even record and reuse.
Once your Tracker is Calibrated properly and you added your tracking solution to the list of tracking devices. You can add them as indexed device or select them in our tracking Compounds and Modules.
Sending Control Data to PTZ system
A PTZ can receive commands from a controlling device or automatically control itself based on its programming. There are a wide variety of hardwares and softwares out there for this task. Generally there are two type of controls.
- controling device can send precise PTZ information and the camera can go to that location.
- the controling device only sends a command for the PTZ camera which itself decides if it can fulfill that command if yes it does. (in example you cannot zoom more than the maximal Zoom. or you cannot rotate more than +-180 degrees.)
Aximmetry does not support Sending PTZ Commands and does not provide any means to control these cameras. Controlling have to be done in
- The camera’s own software
- Separate controlling hardware
- Separate Control Software like NDI Tools/NDI Studio Monitor
PTZ Camera Installation
When you set up a PTZ camera there are some really important aspects which you have to take care of to receive stable and accurate data from the Camera.
It is of utmost importance that the camera is completely leveled. We suggest getting either a stand which itself can automatically level itself. Or to get a Digital Leveler which can provide you complete accuracy when you set up your stand or mount.
It is suggested that the stand/mount of the camera faces the GreenScreen/LedPanels on a Right Angle. As the camera can Pan to any target even if it is not in the center. This can simplify setting up the Virtual Studio a bit.
The stand of the Camera has to be completely immovable. Should be
- fixed to the ceiling or wall or
- rooted to the ground.
If you cant make permanent mounting points and it's on a mobile Platform. Once it is calibrated, It has to be 100% stable and undisturbed from people and forces. As if your camera moves, your calibration will be inaccurate from that moment on.
note: We also suggest taking double attention to cable management. As even if your camera is safe if someone accidentally kicks the cable it can easily move it.
Record Camera Position
Since PTZ cameras are not Complete tracking solutions as they usually can't move without the help of another device. The system in itself only give you rotation information of Pan+Tilt and it doesn't provide Position Data. We need to gather it ourself by hand or by automatic lens Calibration.
In case of all in one PTZ Cameras you only need one information, height position. As these devices sensor position is usually at the PT rotation center. The Height position is measured from the floor. It is usually measured at the Cameras PT Rotation pivot. Aximmetry usually need these information in cm.
In case of Motorized Pan/Tilt Heads
- You need the the height position of the Rotation pivot. (in centimeter)
- You need the relative position of the cameras sensor from the rotation pivot. (XYZ translation in centimeter) (see image)
Using Together with Robotic Legs
Some PTZ manufacturers have additional track-dolly technology to raise the platform it is standing on with an extendable leg and send tracking information. Or a complete Robotic arm working together with the PTZ camera. These are usually pretty unique systems which can communicate a variety of ways. Please always refer to manual or manufacturers infromation regarding the setup of these. Usually the manufacturers have a system to aggregate the different signals into a single data stream per camera.
To minimize latency It is recommended to use a separate network for the tracking data as per your office regular network. This ensures that there is no interference if anyone else uses the network.
In case you only use the network for tracking data you should be able to do it with a single Gigabit connection. (Even though 100Mbps connection might be enough in theory. We suggest to use at least a gigabit connection)
In case of Free-D protocol
Always make sure that your firewall doesn't block either the IP address or the ports of the camera.
PTZ cameras usually use multiple ports to send information to multiple clients at once. By default these ports are in consecutive order. For example on multiple models port 1010 is the default first port. The other ports which these cameras use, are usually: 1011, 1012, 1013.. and the IP might even match. It is not an issue if you use one camera. But if multiple cameras try to communicate on the same port it will create tracking anomalies.
We advise to set each camera's ports distinct from each other. At least 5-port difference , but we suggest to use even more just to be sure.
The reason we suggest to make the safe bet and set each camera to distinct ports is that it ensures no two cameras share the same port and no crosstalk occures.
note: The best case is that all of these ports are exposed in your PTZ control panel and you can edit them individually, but the feedback isthat some systems might only show the 1st port.
Network Setup for Panasonic PTZ Camera
In case you have a Panasonic PTZ camera with Free-D support. After you connected the UTP cable, the DHCP gives an IP for the camera. You can reach the camera's control panel through a web browser.
- Or by clicking the small cogwheel icon at the bottom of the NDI tools.
You have the following views:
For setting up the network settings, you should go to Setup / Tracking Data Output / IP out.
Here you can see 4 possible output Clients. You can turn each of them on or off separately.
Each Client has an IP and a port. It is important to note that the system does not prevent you to use the same IP on different ports.By default these ports are just after each other. It is a common user error to change these IPs to the same IP while having the original ports, while using a second camera set up similarly. While the Client selection should help you select the right setup for multiple receivers, we see that it can cause confusion for some of our clients, and they accidentally turn all of them on. Or forget to turn them off. When this happens, we call it crosstalking. The two systems compete with their data and timing and it manifests in cameras jumping away to another location then back at random frames. Therefore just to be safe we suggest to use port settings which cannot accidentally crosstalk. No two Clients should ever use the same Port. And as there are 4 ports in total we suggest to have at least 5-port difference between cameras.
So if the first port is set to 2000 at CameraA. It should be at least 2005 at the next and 2010 at CameraB.
- 2000, 2005, 2010, 2015 CameraA
- 2100, 2105, 2110, 2115 CameraB.
- 2000 to 2004 is CameraA
- 2100 to 2104 to CameraB.
Tip: This also helps you to remember the port settings as you can number your cameras using the 2nd digit of the port number.
Once you setup your ports in your camera. You should set the selected Client's port in Aximmetry as well. You can do it by going into
Device Mapper / Camera Tracking / Manage Devices... / Camera Tracking / Free-D
Then press the Add button, and add the desired ports for your Cameras, and press OK
Then if you plan to use the Compounds provided by Aximmetry, map them as indexed Camera Tracking devices at:
Device Mapper / Camera Tracking
You can receive video signal on multiple Video Input types.
While professional broadcast cameras can be connected both SDI and NDI.
We suggest to connect your camera through SDI if possible as it can reduce the System latency. Also other devices on a network won`t interfere with the Data transfer. As the SDI has direct connection to the Capture Card and the Machine. While NDI usually goes through our networking devices first.
If your solution has to be NDI due to Distance or technical issues. Make sure your NDI has ample Bandwidth.
Its best if they communicate on a completely separate network compared to what is used for PCs and WiFi in your studio.
Its okay to use the same network as for tracking, however as a video signal sends a bigger package than that of tracking data, You should use at least a tested Gigabit network with appropriate grade cables. Or preferably use a 2.5 or 10 Gigabit network with even better cables. Try to minimize the signal latency by only using as much cable as necessary and not more. Try not to use too many switches or routers inbetween the machine and the camera. In case you use a single camera it might be best to directly connect the UTP to the Computer. (though this needs manual connection setup of that network)
Of course Non-Professional Grade PTZ cameras might not have any SDI connection possibilities, or you use a Photo Camera with mini HDMI output with a PT head and LensEncoder. In this case use HDMI if available as it can send video data to relatively long distances.
Some older PTZ cameras might communicate with DVI or even VGA. Aximmetry can receive these signals as long as you have an adequate Windows compatible capture card for it.
Each camera manufacturer can have its own solution to communicate with the Camera to edit its settings. The two main types are:
- browser based settings similar you find for your router. Where you enter a local network address, and log in to its control panel.
- controlling software/app which you have to install on your machine to reach the cameras settings.
The most common settings which you can find in these menus
- Controls the Networking Parameter of the Camera itself
- IP address
- Controls the camera mechanically/Digitally like
- Controls the video signal which comes from the camera
- Frame Rate
- Image Mode
- Choice of Video Protocol if multiple protocols are available
- Controls the tracking signal which comes from the camera
- Tracking Frame Rate
- Tracking Mode
- Network ports the tracker communicates with the Client apps.
- Lens Calibration Data
- It can happen that the Camera has a built in solution for Lens Calibration, in this case you don`t need to set it up in Aximmetry
- Image Manipulation
- Color Correction
- Built in Effects
When you setup your system for Tracking, it is utmost important that you set matching output and input signal modes. So if you are using 1080/25p on your camera use 1080/25p in Aximmetry for the Camera signal, and 25p for tracking signal.
As most all in one PTZ cameras do not have their own LCD indicator, and control softwares vary regarding how and which information is shared with the user. It is not always obvious what actual settings the camera really has. It is absolutely important therefore that you check these settings in the camera's very own control panel through the network every time you start the system.
The cameras mode can unexpectedly change for various reasons such as:
- Power Down
- Interference from Control Software
- Firmware Update
- Human interference
Therefore we suggest to always check the video mode after powering them up.
To calibrate your camera you should use Aximmetry Camera Calibrator. This software helps you calibrate the camera based on its transformation and Sensor size.
Importantly it helps you to calibrate the Lens itself.
Tipp: Before you start this project. Its best if you place an object, like a chair into your green, so you can track its movement with the camera.
While manufacturers should create a perfect lens… It is not always the case. There might be manufacturing defects. Or some lenses are specially tuned to have their own curvatures at specific angles. To have a better or more artistic output.
By default this is not ideal for a virtual camera, as it always assumes that we are using a perfect lens. We need to provide additional information about these distortions so the Virtual camera can make the necessary adjustments to match the Real cameras recording.
We also need to calibrate the cameras Focus and Zoom information.
How to setup PTZ in Aximmetry Calibrator:
When you start Aximmetry Camera Calibrator, make sure your tracking devices are added in the Device Mapper.
Before starting to record Calibration Points, first you have to set up your Camera Properties.
If you are using an all in one PTZ camera and you set your system up right in the real world, You only have to worry about sensor width and the distance from the ground. As these cameras' sensors are usually at the PT rotation center.
In case you are using a PT head, you also have to add the Delta head postion. Delta head position is the translation (distance) of the sensor compared to the PT rotation center.
Sensor Width is the size of the Camera's light/image Sensor.
To get the Sensor Width please check your cameras manual. Make sure to use a width and not diagonal size. In case its a diagonal size please try to backtrack what is the right size.
Tipp: In case of all in one PTZ cameras the sensor is usually not accessible. However if you are using a PT head with a Camera whic has changable lens, you can simply remove the lens and mesure the width of the sensor woth a ruler.
In case your setup is not perfect, it can still be okay as you can precisely adjust the sensor size at a later point.
PTZ rotation center height from the floor:
The distance from the ground is relatively easy to Determine.
In case an all in one PTZ cameras its usually the height of Tilt Axis. And you don’t need to set up Delta Head transformation as it is generally the same as the PT rotation center.
In case of PT head its also the Tilt Axis but you in this case you have to give Delta Head transformation.
Delta head transformation
You can imagine delta head transformation as the translation vector from the PTZ rotationcenter. It is usually only forward and upward tranformation. As the mounts are usually placed on the Pan axis.
Testing PTZ camera calibration with NDI monitor or Control Pad.
To make a hand calibration you have to be able to check if the virtual objects positions are in match with the real objects positions which the camera records. You can do it by controlling the PTZ camera directly or through pre recorded commands.
The very best solution to test your camera calibration is if you have a physical controller for your PTZ cameras. Using a mixture of Joystick movement with saved camera positions, which has proper time for movement.
In case you dont have a physical controller, you can achieve the same with a PTZ camera controlling software like the NDI tools/ NDI monitor.
Best Practice for PTZ testing
- Recorded States
Most PTZ cameras have the possibility to save PTZ transformation on a control panel to be able to simply animate from one position to another. This is also a really useful tool to test and Hand Calibrate your camera. As these functions allow you to repeat the same process with the same hardware settings, while only the calibration outcome changes. You can easily see the difference between settings to approximate your outcome.
We suggest to use your numpad for changing between state 1 to 9.
Its also vise to setup 1-3 as Max Zoom out. 4-6 as Middle Zoom and 7-9 as Full Zoom from the beginning. As it will speed up setting up your calibration poins.
- Keyboard Control
Alternatively if you use NDI tools to control the PTZ camera or any other software which allows you to use a keyboard to control the camera. It is advise to move up and down from a fix saved position. Or Left and right from a fix saved transformation. This ensures that you only have to worry about a single axis during calibration therefore having a better chance to see which parameters need to be adjusted.
- Joystic and Mouse Controls
Last but not least you can directly control your camera with your joystick or mouse, this allows complete a freedom of motion. And this is genuenly used as a final test, to make fine adjustments at the end if neccessarry.
Once you have setup your basic properties and you are familiar with controling your PTZ camera. You should setup your tracking delay. Video and Tracking data comes on different protocols, and go through different processes. Therefore they have a bit of time difference in the system, which we can simply sync by providing a tracking delay value. Tracking delay is calculated in frames.
To setup tracking delay, you should make a sudden move with your camera and see on the screen how much later (or sooner) the 3D environment moves compared to the cameras image. Write in your perceived time value (in frames). Then make another fast movement. Adjust the delay value, then repeat the movement/adjustment process till you feel your 3D and Camera image is starting and stopping their movements at the same time.
Tracking delays can change everytime you start your system. So we suggest to adjust your tracking delay values before every show.
To have a better understanding of what is happening during the calibration, Aximmetry provides a checker textured floor, and a set of Marker 3Dobjects
You can place these Markers by editing its position in the menu. The best practice if you can place them to the legs of the chair we suggested to use earlier. As it is really easy to see if the two object is at the same position after rotating the camera.
To imagine what Calibration Points do is the best if you picture a graph. This graph is responsible to adjust the distortion differences at specific zoom points. In default the Aximmetry uses the prameters of an "ideal" lens. But these are only default values not for a specific lens.
Usually we take the first calibration point where the camera is completely zoomed out. This gives a baseline Focal Length and Horizontal Field of View. (Also other fine tunings)
To setup this value you should use the above mentioned Camera motions to move the camera around. See if your Virtual Marker fits on the Real World Marker. Adjust the Focal Length and Field of view till it matches completely.
Next if possible we should zoom in Completely and adjust the calibration there as per mentioned previously.
With this we are having a linear transformation between the two endpoints.
At this moment you should see how accurate is your zooming, and test if Zoom interpolation Mode 1 or Mode 2 seems to be closer to a right movement. (Depending on your camera it can differ)
So after deciding on the mode for now.
You should adjust camera calibration as per described at Half (Mid) Zoom. Then see if your Zoom mode can still apply.
Usually 3 points is not enough for a perfect calibration. So you should set up the
- Quarter Zoom
- ThreeQuarter Zoom
And after this point make multiple Motion and Zoom tests to see if these points are comletely accurate or not..
If the 5 main point is as accurate as can be but you still see imperfections in the inbetween steps. Its best to just add more points inbetween as you feel neccesarry. But always make sure the first 3-5 points are 100% accurate as it saves time and hassle for you later.
Zoom Interpolation Mode 1 and Mode 2.
Once you setup your Calibration Points. You might see that between the points the Camera does not calculate Zoom properly. As some camera uses different mechanical and optical behavior to other. For this case you should try out Interpolation Mode 2 instead of 1. And use the one which is better suited for you.
Receive Tracking information in Aximmetry.
To receive tracking information in Aximmetry. You have to add your tracking devices at the startup/Devices/Manage Devices/Tracking Devices
What to look for when choosing a PTZ Camera.
There are multiple solutions to achieve a “PTZ Camera” on the market.
Cheaper models tend to be closed systems which use their proprietary protocols and do not provide tracking data to third party softwares. These are usually sold as security cameras.
While some more expensive professional PTZ cameras have the ability to connect to third party softwares and send tracking information through open protocols like the commonly used “Free-D” protocol. These cameras are usually sold for Studio and Scientific purposes.
note: There are some extremely rare cases when you can find nonProfessional Cameras providing tracking features on an open protocol. However usually these cameras’ specifications are not clear or consistent. We suggest to always inquire information about tracking capabilities through the channels of the Manufacturer before purchasing.