Introduction
With Augmented Reality, we can enhance our production by adding virtual elements to the surroundings of the talent. These elements can be static, or mobile virtual objects, lights, virtual screens, charts and bars for statistics, and even other talents located in distant studios. Usually, these elements look realistic and their purpose is to enhance the viewer experience. With the help of AR, the illusion that digital objects exist in the real world and interact with them can be created.
AR elements can be added in post-production to recorded footage, but since Aximmetry was made with broadcast productions in mind, we will discuss the case when AR elements are added during the broadcast, in other words in real-time.
Basically, it is the opposite of the previously mentioned Green Screen and LED-Wall productions, since in those we have placed the real-life talent into virtual reality, meanwhile with AR virtual elements look as if they were placed in reality.
In the example below, you can see that a 3D virtual model of a racecar is added to a real office space.
NOTE: In this article, we do not consider overlays, and lower thirds like crawls and channel logos AR.
Why Should We Use AR in Virtual Production?
AR is commonly used in broadcasting production environments mostly for the reason that it is one of the most versatile ways to add information to a production.
It is an incredibly useful tool for showcasing sports, gaming, entertainment, medicine, education, business, and architecture, to name a few examples. You can display infographics, data visualization, meteorological information, etc.
In the example below you can see a map that shows the metro reconstruction sites in Budapest:
Another example shown in sports broadcasting is the virtual display of players’ positions on a real football field:
AR Workflow
The steps of an AR production are simple:
1. - The studio camera sends its image to the PC running Aximmetry. At the same time, the camera tracking system forwards the data of the physical camera's movements to Aximmetry.
2. - Aximmetry places the AR elements onto the studio camera's image using the camera tracking system's data to match the perspective of the rendering to the physical camera's.
3. - Aximmetry outputs the final image.
What Do You Need for an AR Production
AR can be added to any camera's image, but for broadcast production, it is generally used in a real studio environment. For example a newsroom studio or a studio for weather forecasts.
AR can be used with tracked or stationary cameras. Now we will go through the 2 methods with all the pros and cons.
NOTE: Discussed here are the minimal requirements only. You might want to add certain hardware to your setup, a Sync Generator for example.
AR with Stationary Cameras
AR with a stationary camera is simple, however, it has its limits.
It needs:
- Aximmetry
- a camera
Its advantage is the simplicity but you cannot move or zoom the camera because it is untracked (the AR elements will not be able to react to the changes of perspective the camera may have).
AR with tracked cameras
It needs
- Aximmetry
- a camera
- a tracking system
The camera tracking system tracks the studio camera movements and sends this data to the virtual production software (Aximmetry). The virtual production software will change the perspective of the AR graphics to match the perspective of the studio camera, making the AR elements look more like part of the physical studio.
The advantage of this installation is the freedom of movement and zoom.
Disadvantages may be the additional cost of the tracking system, the additional time spent on calibration, and the more complicated setup process of the scene.
Here you can learn more about setting up an AR production:
