Introduction to LED Wall Production

Introduction

This document discusses the basics of LED Wall-based virtual production, focusing on five aspects: studio personnel, hardware, rendering, output, and post-production.
NOTE: This document supposes that you are already familiar with the basics of Aximmetry and the use of camera tracking systems. If not, then we recommend first proceeding with the following documents:
Introduction to Starting with Aximmetry
Introduction to Tracking

Studio Personnel

LED Wall-based virtual productions use special components, namely the LED Walls and the LED processor.
The LED Walls are used to display video content (e.g., virtual studio scenes rendered by an Aximmetry computer).
The LED processor controls the resolution, frame rate, and output of the LED Walls.
Both components require in-depth knowledge, usually provided by an LED Wall technician.

NOTE: This documentation will not cover the setup process of these components.

Hardware

Hardware Connections

We recommend connecting to the LED processor from your GPU using HDMI/DP cables, as these protocols support arbitrary resolutions that are often suitable and required for LED walls.
You can also use the SDI output of your capture card; however, this approach requires pixel mapping in your LED wall processor, as the SDI protocol only supports specific resolutions.

NOTE: You can use NVIDIA RTX PRO, RTX Ada, or an RTX A-series (formerly known as Quadro) GPU paired with NVIDIA PRO Sync cards (formerly known as Quadro Sync II) to add a sync to your output. This is the recommended line of GPUs for LED Wall-based virtual production. See: 
Genlocking

Camera Tracking

You can use stationary/fixed cameras with limited functionality, but it is strongly advised that you have a camera tracking system already set up and configured for each camera you use.

Tracking information is used for four different purposes:

• to render the virtual graphics (displayed on the LED walls) from the perspective of the studio camera
• to help with specifying the LED wall positions in the 3D virtual space
• to render the Digital Extension around the LED wall's position.
• to place AR graphics in front of the camera picture.
  NOTE: To learn more about this, please refer to the following document:
  Combine Different Productions in Separate Machines

Genlocking

The control of Delays plays a crucial part in the success and quality of LED Wall-based virtual production; therefore, we must have absolute control over them.
Use a genlock device to genlock all your devices:

• Cameras
• Camera Tracking Systems
• LED Processor
• GPUs
  • NOTE: To genlock a GPU card, you'll need an NVIDIA RTX PRO, RTX Ada, or an RTX A-series (formerly known as Quadro) GPU card and NVIDIA PRO Sync cards (formerly known as Quadro Sync II).
    
  • NOTE: You can use other types of GPU cards too, but without being able to genlock your GPU card, thus certain features may not work ideally.

IMPORTANT: Make sure that all your devices use the same frame rate (including the Genlock device).

Rendering

Render Settings

To provide the correct image content for all LED walls, Aximmetry performs the following render passes:

• "Frustum" rendering: the image portion that the camera sees on the LED wall. This should be done in the highest necessary resolution and quality.
• "Fill" rendering: one pass for each LED wall, they provide the content for the parts that the camera does not see (due to them being outside the field-of-view of the camera). Since the main purpose of these image parts is to provide ambient lighting and reflections, they can be rendered in reduced quality.
  

NOTE: Both the Frustum and Fill can independently be adjusted in many ways. To learn more about this, please refer to the following documents:
FILL Adjustments
FRUSTUM Adjustments

Number of PCs Needed

Depending on the number of LED walls, the Frustum's image resolution and frame rate, and the resolution reduction of the Fill passes, you might find a single computer enough to render all the content.

Depending on the production requirements, you may find that you need multiple computers to achieve them. Such requirements can be (among others):

• The number of LED walls/processors
• The number of cameras
• The use of Digital Extension combined with other requirements
• The use of Animations combined with other requirements

If you need multiple computers:

• IMPORTANT: The "Frustum" pass always has to be performed by all computers.
• In some cases, it is necessary; in others, we strongly recommend the use of an additional computer to act as the control machine.
• If you need very high-resolution rendering for all the Fill rendering, you might need multiple computers to perform that. In this case, you have to specify which LED wall is rendered by which computer. 
• We recommend using one computer for each LED wall to achieve better performance and visuals. Additionally, we suggest splitting large LED walls into at least two walls, each rendered by a separate computer.
  

NOTE: The use of Digital Extension is limited when using a single computer for LED Wall Production. To learn more about single-machine LED Wall production-related limitations, please refer to the following document:
Single-Machine LED Setup

NOTE: You can learn more about the number of computers needed for your production here:
In the Case of LED Wall Production

LED Wall Shapes in Aximmetry

Aximmetry supports the following LED Wall shapes:

• a flat rectangle
• a flat triangle
• a curved rectangle (where the curve is an arc of a regular circle)
  
• a curved triangle (where the curve is an arc of a regular circle)
• custom shape masks (requires an image file with an alpha channel to determine the shape)
  

NOTE: The above-listed shapes can be combined within a single production, specifying one option per LED Wall.

Virtual Content Stages

The virtual content goes through 3 stages in the case of real-time LED Wall-based virtual production:

• The LED Wall output (the virtual scene displayed on the LED Wall(s) provided by Aximmetry Composer)
• The composite image (in the case of an LED Wall-based virtual production, this is the image captured by the studio camera)
  
• Post-production (Aximmetry provides real-time post-production features to further enhance the production)

LED Wall Output

Aximmetry renders the FRUSTUM image (the virtual studio scene from the perspective of the selected camera) and also renders the FILL image (used for ambient light and reflections), then outputs the image to the LED processor (which then transmits it to the LED Wall).

NOTE: The virtual background is always projected onto the LED walls from the tracked position of the currently selected camera.

Composite Image

You can shoot the scene using a single or multiple studio cameras.

The signal coming out from a camera is already the final composition of the real foreground and the virtual background when using LED Walls. Therefore, you can record the picture on the camera itself or lead the signal directly into your studio equipment for broadcasting/recording.
IMPORTANT: The content on the LED Wall by default is always rendered from the perspective of the selected camera. In the case of using multiple cameras with default rendering settings in Aximmetry, only one camera at a time captures a correct perspective. Aximmetry, however, can be configured in a number of ways.

NOTE: While this stage already provides production-ready results (that can be broadcast to e.g, a live audience), we do recommend first feeding the camera output back to Aximmetry to take advantage of the real-time post-production features.

Post-Production

There are a few reasons why it is recommended to wire the signal back to the Aximmetry computer:

• The real-time digital extension only works if you feed the camera's output back to Aximmetry.
• It is much easier to set up the position of the LED walls while watching the camera picture. This is the preferred way.
• Pre-viz keying can only be done this way.
• You might want to record the final content with Aximmetry.
• You might want to make adjustments or do any post-processing on the image.
• You might want to composite further elements on the image, like overlays, PIP, text, channel logo, etc.

Below, you can learn more about some of the above-mentioned features.

Digital Extension

Digital Extension is a feature that allows extending the virtual studio scene beyond the physical limitations (edges) of the LED Walls. This allows the virtual camera to capture the virtual studio scene even when the studio camera captures the surroundings of the LED walls.

NOTE: Example of an LED Wall's corner captured by the camera without (left) and with (right) Digital Extension.

While it is commonly thought to be part of the production (thanks to the advantage of live-rendering), from a technical perspective, Digital Extension is considered to be a part of the post-production.