Mobile Device Tracking Setup - LiveLink

Section 10. Mobile Device Tracking Setup - LiveLink

Currently, there are two established methods to use Live Link for virtual production that we've tried and tested: Vive Mars and with the Live Link VCam app (iOS only)

Section 10.1 Live Link VCam App.

This is a free app on Apple's App Store made by Epic Games. It uses the ARkit, your cameras, and the device’s gyro to perform tracking. As of now, this is the only way to get positional tracking needed for our tech to work.

With the Unreal VCam app, you can achieve virtual production-ready cinematography directly from your iPad or iPhone in real time. The app allows you to seamlessly translate traditional filmmaking techniques like camera lensing, framing, and movement into Unreal Engine's digital space.

You can record realistic camera takes, refine them within Unreal Engine's Sequencer, and review scenes during all stages of production—from pre-visualization and principal photography to post-production.

The app supports ARKit for streaming data over a network, driving 3D cameras in real-time environments, and integrating camera views with accurate depth of field for more immersive experiences.

Additionally, it offers features for camera parameter adjustments and timecode synchronization, ensuring precise and efficient workflow integration.

Requirements

• Live Link VCam

• iPhone, iPad, or iPod Touch running iOS 15 or greater

• Your host machine's local IP address.      

Section 10.11 Setting up the Unreal VCamp App.

Once installed and opened, you will presented with a very simple screen asking for the IP address of your host machine. To find this address do the

following:

1. In Windows 10/11: Open the Windows settings page.

2. Click on Network & Internet > Advanced network settings.

3. From the Advanced network settings page, click on Hardware and Connection properties

4. Scroll down to Properties to find "IPv4 address".

5. Enter the value there as you see it.

Important

Don't connect yet!

6. Follow the instructions for Live Link for the in-scene Options Menu.

Section 10.12 Choosing Your Tracking Method.

If everything is set correctly, then you'll be presented with the above image. Choosing either option will immediately spawn the camera you

desire. However, if you have not done the proper setup, you may instead get a selection as follows:

If everything is set correctly, then you'll be presented with the above image. Choosing either option will immediately spawn the camera you

desire. However, if you have not done the proper setup, you may instead get a selection as follows:

This will mainly show if you do not have UDP Messaging enabled (editor) or have the proper flags set via shortcut. Make sure you've followed the

steps above to fix this issue!

Section 10.13 Options Menu.

Once the scene is running with your desired tracking method, you will use the Options menu to configure it further. You can access this menu by either pressing Escape or Backslash on your keyboard.

This menu only shows when using SteamVR as your tracking method, and its use is pretty simple.

Section 10.14 Select Tracking.

This brings up the UI to help you choose your tracking method again.

Section 10.15 Patch Steam VR.

This button will only be available if Unreal detects that your install of SteamVR is configured to require the use of a Head-Up Display (HMD) to be

used.

This button will change the settings of SteamVR to disable this requirement and the UI will change to report if this task was successful or not. If

If it is not successful, a message will appear detailing what you may need to do for this function to work.

Section 10.16  Auto Buttons.    

This will poll SteamVR for any and all VR trackers and controllers that it has detected. The serials produced are in the order that they were connected,

so be sure to connect them in the order you want.

Should this button not find anything, you will receive a brief message asking you to run SteamVR and connect your desired trackers before trying

again. If you still cannot get any serials to show up, you may need to restart your project/scene.

Section 10.17 Livelink.  

When you first open the Options menu with Live Link tracking, you'll be greeted by this initial window. If you are using the Vive Mars or Live Link

VCam and they have the proper IP address set, you should eventually see the menu populate itself with available Live Link clients for you to use.

Select Tracking

This brings up the UI to help you choose your tracking method again.

Refresh

This will force the UI to search for available clients. If it finds any it will populate the menu with a list of client sources.

Section 10.18 For Live Link VCam Users.  

Go ahead and press connect on your device. You will see that not only does it connect to your PC, but it will also stream your screen in real time! The app will ask that you move the device around in order for it to gain its bearings. Follow its prompts and continue on to Mobile Device Tracking Setup below

Disconnect  

Disconnects your connection to the device. It will switch to allow you to reconnect to the device as needed.

Set Hero.

Will set the device in the list as your Hero Tracker. Doing so will clear the device from being used as your Calibration tracker.

Set Calibrator.

Will set the device in the list as your Calibration tracker. Doing so will clear the device from being used as your Hero tracker.

Section 10.19 Mobile Device Tracking Setup - Live Link.  

Setup Steps

1. Make sure to set your device as the Hero tracker (click on Set Hero)

2. Enable Use Hero Tracker as Calibrator

3. Enable Flip X-Axis.

4. Save and exit the options menu.  

5. Press the Tab to bring up the Calibration screen.    

6. Hold your iOS device in portrait mode (or the way the screen is being presented on your device) and try to center your device along the

crosshair on your screen.

7. Press C to calibrate and pull your device back.  

8. It should now be tracking correctly.            

Section 10.20 Tracking Position Troubleshooting.  

Make sure Lock Tracking Rotations is disabled:

• If, when you move the device forward/back, you get the opposite movement, enable Flip X-Axis.

• If, when you move the device right/left, you get the opposite movement, enable Flip Y-Axis.

• If, when you move the device up/down, you get the opposite movement, enable Flip Z-Axis.

• If, when you move the device forward/back, you get up/down, enable XYZ to ZYX Coordinates.

• If, when you move the device right/left, you get up/down, enable XYZ to XZY Coordinates.

When doing any of the above, be sure to disable all the other Tracking Modifiers so you do not get confused.

Set Calibrator:

This will set the device on the list as your calibration tracker. Doing so will clear the device from being used as your Hero tracker.