Fixing or Alleviating Tracking Jitter

The Tracking Volume.

Imagine this as our physical tracking space. We have our screen and a camera, but no tracking solution yet.

The tracking volume is the space in which your tracker and its equipment are able to orientate and locate  themselves within its bounds. Whether it is by bathing the area in IR, a camera searching for IR reflective star patterns, or even by CV sensors, the end result is an output of 6DOF data that can be passed into your host computer and eventually into Unreal from which we can use said data.

In a perfect setup, there should be nothing between the tracker and the sources it uses to orient itself within the volume. Unfortunately, this is never the case in virtual production.            

The First Major Issue: Lighting & Fixtures.

Or, more importantly, the types of lights used on set. Most tracking solutions make exclusive use of infrared cameras to track in 3D space, and having lights that lights that can output into that spectrum will affect tracking drastically. You’ll often see this as jitter or sudden jerks in position even when the tracker is sitting still. The type of lights you use and the placements relative to the tracker on your camera are important as you do not want your tracking to be compromised while filming.  

Important

The biggest light source of all is also a virtual production’s biggest killer: the sun. Nearly all tracking solutions cannot work if they have sunlight coming in due to IR interference.

An example of lighting done properly in one of ARwall’s affiliate work using SteamVR.

The Second Major Issue: Blocking the Tracker’s Source.

The second and most common issue is blockers that prevent the tracker from “seeing” its tracking source. Most, if not all, tracking solutions use a camera of some kind, and having too many objects or people in the way of that camera and its tracking target can easily affect the tracking quality. This all depends on the style of tracking your particular solution utilizes.    

If you are using the Base Station style tracking like that found on the [ViveMars], then you simply need to ensure that the tracking puck has a clear line of sight to the base station at all times when considering the amount of movement you’ll likely perform (the camera is in the base station). If your tracking solution has the camera within the tracker itself, then you will need to have a clear view of whatever star pattern the system uses (which is usually the ceiling or floor).

Keep in mind that regardless of which tracking solution you use, the cameras that they often utilize have a wide field of view or multiple cameras to help orient the tracker within their space. As long as you have a clear line of sight, the tracking system should have no problems.

In this moving shot we had to specifically space our base stations in a way so that the trackers could be seen at all times.

An Issue Worth Mentioning: Reflective Surfaces.

This is especially an issue with tracking solutions that blast IR into the volume, like with the base stations in the ViveMars. Because of the way it floods the area with IR, any reflective surface can and will bounce and scatter that light, causing a cascade of light interference. As an example, within a demo booth, there was a window that was blocked off but on the other side.

Even though the light was not coming in from the outside, this window was still exposed to the tracking volume side, causing our tracking to jump and skip when moving. Blocking reflective surfaces like this is important and may have to be done on more than just windows and mirrors.

Another Way to Alleviate Jitter Specifically for SteamVR: Thresholds.

The following is an excerpt from Track_SteamVR. Please check it out for more information on thresholds for both ARFX Pro Plugin and [Scenepack]s.

Thresholds.      

These base settings have worked in most stage setups, but depending on your stage, these need to be increased.

These settings are important to the stability of your tracking. “Tracking jitter” is the term we use when we describe a particular problem that occurs when the tracking location and orientation tend to shake or vibrate even though the tracker is sitting perfectly still. This problem is most prominent with orientations as minute variations in this data are magnified significantly on whatever is attached to this data (particularly when using SteamVR on a normal green screen Virtual Camera, you will see the view shift randomly depending on your tracking environment).

The solution to this problem was to implement velocity thresholds, the minimum amount of movement the tracker needs to make to be considered usable. The data is super precise. Thus, the values you are actually changing are the amount of precision to use in its checks.

• Disable Velocity Thresholds - This completely disables all velocity thresholds. Not recommended unless it is for troubleshooting.

• Velocity Power Threshold - Linear movement threshold. The amount of precision used to check the tracker’s velocity data against. The lower the value, the harder it will be to break the threshold.        

• Angular Velocity  Power Threshold - Rotational movement threshold. The amount of precision used to check the tracker’s angular velocity data against. The lower the value, the harder it will be for the rotational speed to break the threshold.        

Important

Power thresholds are actually the value a negative power. In other words, 10^-X. So by default, ARFX's SteamVR Component uses 10^-14 or 0.00000000000001 as the threshold for movement!

When to Change Your Thresholds.                                                

If you consistently see jitter or random movement when the Hero Tracker is sitting still, then you may want to investigate your thresholds. However, it is best to look to your tracking volume first for anything that can cause a disruption of your base station’s IR signal. Check out How to Setup Room/Stage Scale Tracking section for information on major issues that can arise within your tracking volume.

If these issues do not pertain to your tracking volume, then you will have to use the above thresholds to help mitigate your tracking jitters. Lower the value one point at a time, checking to see if you still have full movement of your hero tracker. Stop when you’ve found a value that allows you to continue movement while mitigating or eliminating disruptive tracking jitter that you had before.

Alternatively, if moving the tracker does not fully register the movement, or it only registers movement when moving fast, then you will have to increase the value of the thresholds, making it easier for the speed of the tracker to be registered as an allowed movement.