How to Setup Room/Stage Scale Tracking

Section 1. How Setup Room/Stage Scale Tracking

The physical setup of your tracking volume is crucial to the setup and operation of not just ARFX but on any virtual production methods you’ll use. Traditionally, these tracking volumes are designed to be able to track as much space as possible so as to fit the needs of any potential use case.

They are often fixed in place and cannot be moved once calibrated, which can be both good and bad depending on the particular needs of your shot. Each tracking method has its own strengths and weaknesses, but your ultimate goal is to track your camera and screen’s exact position reliably and without hassle.

In this section, we will cover a generalized method of setting up your tracking volume for the most common tracking solutions in use with our ARFX Cameras.

Because this is only an overview, we will not be going into highly technical details. For that, we suggest looking into the specific section after this  section and your particular tracking solution’s documentation.

Section 1.1 The Tracking Volume.  

Imagine this as our physical tracking space. We have our screen and camera but no tracking solution yet.

The tracking volume is the space in which your tracker and its equipment are able to orientate and locate  themselves within its bounds. Whether it is by bathing the area in IR, a camera searching for IR reflective star patterns, or even by CV sensors, the end result is an output of 6DOF data that can be passed into your host computer and eventually into Unreal, from which we can use said data.

In a perfect setup, there should be nothing between the tracker and the sources it uses to orient itself within the volume. Unfortunately, this is never the case in virtual production.

Section 1.2 The First Major Issue: Lighting & Fixtures

Or, more importantly, the types of lights used on set. Most tracking solutions make exclusive use of infrared cameras to track in 3D space and having lights that can output into that spectrum will affect tracking drastically.

You’ll often see this as jitter or sudden jerks in position, even when the tracker is sitting still. The type of lights you use and the placement relative to the tracker on your camera are important as you do not want your tracking to be compromised while filming.

Important

The biggest light source of all is also a virtual production’s biggest killer: the sun. Nearly all tracking solutions cannot work if they have sunlight coming in due to IR interference.

An example of lighting done properly in one of ARwall’s affiliate works using SteamVR.

Section 1.3 The Second Major Issue: Blocking the Tracker’s Source

The second and most common issue is blockers that prevent the tracker from “seeing” its tracking source. Most, if not all, tracking solutions use a camera of some kind, and having too many objects or people in the way of that camera and its tracking target can easily affect the tracking quality. This all depends on the style of tracking your particular solution utilizes.        

If you are using the Base Station style tracking like that found on the ViveMars, then you simply need to ensure that the tracking puck has a clear line of sight to the base station at all times when considering the amount of movement you’ll likely perform (the camera is in the base station). If your tracking solution has the camera within the tracker itself, then you will need to have a clear view of whatever star pattern the system uses (which is usually the ceiling or floor).

Keep in mind that regardless of which tracking solution you use, the cameras that they often utilize have a wide field of view or multiple cameras to help orient the tracker within their space. As long as you have a clear line of sight, the tracking system should have no problems.

In this moving shot, we had to space our base stations specifically so that the trackers could be seen at all times.

Section 1.4 An Issue Worth Mentioning: Reflective Surfaces

This is especially an issue with tracking solutions that blast IR into the volume, like with the base stations in the ViveMars. Because of the way it floods the area with IR, any reflective surface can and will bounce and scatter that light, causing a cascade of light interference. As an example, within a demo booth, there was a window that was blocked off but on the other side.

Even though the light was not coming in from the outside, this window was still exposed to the tracking volume side, causing our tracking to jump and skip when moving. Blocking reflective surfaces like this is important and may have to be done on more than just windows and mirrors.

Section 1.5 Setting Up Your Tracking Volume for ARFX.

The key to setting up tracking for ARFX is to include your screen within that volume. To calibrate ARFX, it first needs to know where in virtual space that screen is located and its orientation in regard to your hero or camera tracker. All movement of the Hero Tracker after this is done relative to this point in space. Without this, you will have to resort to alternative means of calibrating your screen’s position via Calibration Offsets.      

Section 1.6 Example in Base Station Style Tracking Solutions.  

In the case of having two or more base stations for SteamVR or Vive Mars, you must be sure to have at least one base station oriented so that the screen can be seen within its 120° field of view. It is also important that each base station can see at least one or more of the other stations within the volume. When expanding the number of base stations, it is important that you maintain a visual line of sight between stations so they can identify and communicate properly.

This is a representation of a physical setup, not one you do for real. Note that this is not a full representation, as each base station has a 120° field of view!

A top-down view of the above example setup. Two important notes: they will always be able to see each other, and one tracker must include a view of the screen’s surface.    

The goal is to make sure the Trackers will always be seen by at least one Base Station regardless of where it is in the intended tracking/play area.

Each Base Station should be placed at a height of around 2 meters/6.5 feet tall (light stands with proper adapters are recommended, but Base Stations can be attached directly to higher walls, ceilings, or set pieces if needed – just make sure that whatever the base stations are attached to is not easily disturbed, moved, or shaken during filming).

Each Base Station field of view should be tilted slightly down toward the intended tracking/play area.

Each Base Station field of view should slightly overlap the other Base Stations’ field of view and all Base Stations together should cover the tracking area that you expect the Trackers to be active in. Keep in mind that each Base Station has a 120-degree field of view, but with creative daisy-chaining, a few Base Stations can create a sizeable play area for the tracked camera.    

Be sure that at least one Base Station can see the screen that you are using (the center of the screen, where you will be calibrating with a tracker).

Make sure that Base Stations and tracking/play areas are away from sunlight or other reflective surfaces such as mirrors or glass if possible. These factors can affect tracking and should be minimized as much as possible on set.

Some onset lights can cause IR radiation to interfere with tracking stability with Base Stations. It is recommended to turn one light on at a time while testing tracking to find out if lights are causing tracking issues. The most common lights that cause issues are older ones.

For more tips on base station placement, check out Vive’s own ViveBaseTips.                        

Section 1.7 Example in Star Pattern Style Tracking Solutions (Stype, Mo-Sys, Antilatency).  

Again it is important to have your screen within the volume that your particular tracking solution uses. Most, if not all, the manufacturers that use this tracking solution allow for the expansion of the tracking volume should yours not be large enough to cover where your screen is located (however, each one has vastly different ways of recalibrating the system to accommodate for the new pattern). Once you have that done you are ready to go. There are hardly many considerations that need to be made for this setup over what has been covered before.

This virtual representation of a physical setup is pretty straightforward. If you cannot move the screen to the tracking volume, then it would be best to move the tracking volume to the screen. Having trusses like shown has proven to be invaluable on an ever-changing stage.

Section 1.8 Using Calibration Offsets When You Cannot Move the Screen to the Volume

A BTS photo of a stage we had the honor of working on with ARFX.

Clearly, there was no moving this massive stage towards the screen!

There are times when you simply cannot move your screen closer to your camera for several reasons. What do you do then? On each ARFX Camera’s Details panel, there is a Calibration subsection within ARwall that has the following setting: Screen Offset.

An important setting that is for calibration only.                      

In order to use this, you must do the following while you have the calibration screen up while in play:

Find a spot within your tracking volume that will be fixed and used for calibration. If you have a calibration tracker, then this can be anything that you know will not move, such as a point on the floor. If you only have your hero tracker, then use a fixed point on the set that you can reuse whenever calibration can be used.          

Measure the distance from the calibration point to your screen in centimeters and be as straight as possible (even if it is not directly to the wall itself).

Measure the distance horizontally from the calibration point to the vertical centerline of the screen (be as direct as possible without height deviation).

Measure the difference in height from your calibration point to the screen’s center point where the crosshairs meet.

Input your findings into the settings above in order in centimeters (X, Y, Z). To know whether to go +/-, use the following image:

So if you are 317cm away from the screen, 32cm to the right of the screen's center, and 100cm below the center of the screen, you input (-317, 32, -100).