CineCam Lens Component

Section 9. CineCam Lens Component

With ARwall’s Cine Camera Lens Component, you can drive a LensEmulation-enabled ARFX Camera through an external Cine Camera Actor of your choosing. The reasons for doing Lens Emulation this way are many but it boils down to these major points:

• We do not want Lens Emulation to change values that could affect the final ARFX image warp.

• We do not want any accompanying motion tracking data to affect the position of our ARFX Cameras

• Several data points would not work anyway, such as with Focal Length.

• We want you to have the freedom to use other tools in Virtual Production, such as Unreal’s Lens Calibration Toolkit.

• To maintain cohesion in the setup process similar to that of the Track_Universal.        

Lens Emulation is not a hard requirement for ARFX, but it is nice to have. In the times it has been called for, most often, we’ve found that filmmakers like their backdrop to have either a set amount of defocus or none set at all. Instead, they tend to rely on their own physical camera to do most of the actual work.

Section 9.11 Component Details

There is not much at all here, as the rest can be found on Lens Emulation-enabled ARFX Cameras.

Camera Reference - The reference link to the Cine Camera Actor found within the scene that this lens component will pull its lens data from. Using the drop-down menu will show the complete list of all Cine Camera Actors in your scene. Alternatively, you can use the eye-dropper tool to click on the specific camera you wish to use.

Use Manual Focus Distance - The component will try to pull from the selected Cine Camera Actor’s Manual Focus value instead of Current Focus. This setting is enabled by default and it is recommended to keep it enabled. Current Focus is often not used or has its timing behind the actual real value, especially in recorded sequences. use manual focus distance.  

Section 9.12 Where are the rest of the settings?

You can find any and all settings related to LensEmulation on the ARFX Lens Camera itself. For more information, check out the ARFX Lens Camera Section

Section 9.13 Setting Up the Cine Camera Actor for Lens Emulation - Live Link Lens

In order for the Cine Camera Lens Component to obtain a live feed of your lens data, you need to have a Cine Camera Actor with Lens Emulation attached to it. In this section, we will briefly go over how you can do so with Live Link Lens. If you want a full explanation of how to set up Live Link for motion tracking, then please refer to the Setting Up and Using Live Link or FreeD Section.

Section 9.14 Required Hardware

Many hardware tracking solutions do not come with a way to send lens data in their base products. The hardware for doing so is usually an add-on that must be purchased separately. An Example would be the FizTrack, which uses a series of gears and a servo that report whether the lens is at its minimum (0%) or its maximum (100%).

Only a select few have a means of obtaining the data directly from the lens itself without using interpolation, and ARwall is one of those very few, albeit with only one Canon Compact-Servo 18-80 (which you can learn about in Canon Lens Component).

Some lenses have free infinite rotations, and this style is not fully compatible with many of these lens addons as they require a clear start and end to work correctly. A workaround is to find the range you want and prevent the lens from going further (like using zip ties).

Section 9.15 Required Plugins

These plugins are necessary for Live Link and Live Link Lens.

In addition to the above, you will also need a plugin called Camera Calibration. This may ask that you also enable OpenCV Lens Distortion. Allow it to do so.

First, you will need to enable the plugins as pictured above. To do this, you need to open the Plugins Browser Tab by clicking the Settings Icon > Plugins

On the plugin’s browser, search for Live Link.

Enable Live Link and Live Link Lens. Optionally, you can enable  Live Link FreeD if you are using that protocol.

Search for Camera Calibration or Lens

Enable Camera Calibration and OpenCV Lens Distortion. Note that enabling Camera Calibration might immediately ask to enable OpenCV Distortion as well. Allow it to do so.

Important

Once everything is enabled, you may be asked to restart Unreal Editor. Be sure to do so!

Section 9.16 Set Up Live Link

You can follow the instructions on how to do this in the Setting Up and Using Live Link or FreeD Section. Although the article does not specifically cover how to set up the lens emulation data stream, if you have the hardware and it is setup to use it, then simply streaming Live Link or Free-D will do half the job for you. The rest is setting Unreal to actually make use of your lens data stream.

Section 9.17 Setting Up Your Cine Camera Actor for Lens Emulation

If you have followed the Setting Up and Using Live Link or FreeD Section, you may already have a Cine Camera Actor in your scene with motion tracking setup. If you have not, please do so and then return here for your next steps.  

Start by adding the Lens component to your Camera.

With the Cine Camera Actor selected, use the +Add button in the Details menu to bring up the Add Component menu.

Search for _Lens_ and add Lens.

Select the newly added Lens component in the component list.

The Lens component’s detailed settings list. Your focus will be on the highlighted settings above.

Make sure Evaluation Mode is set to Use Live Link

In Lens File, use the drop-down menu to create a lens file by selecting Lens File at the very top.

Back on the Live Link Controller you will have to set the Lens file as well.

Select the Live Link Controller.

Navigate to Role Controllers > Camera Role > Camera Calibration > Lens File

Set the Lens File to the same one you created in the Lens component before.

Section 9.18 Setting Up Your Lens File in Unreal’s Camera Calibrator

The Camera Calibration menu in the Unreal Editor. Typically, this is where you calibrate Unreal for lens distortion, focus, focal length, and more. We care for only Focus in ARFX.

By default, Live Link uses a generic lens graph for Lens Emulation. To fine tune your lens you have to manually map out the values that Unreal receives and assign to that the actual value that your lens is at.

To access this menu, you will need to find the Lens file you created in the steps above and open it, bringing this menu onto your screen. Initially, it will look like the image above.

Be sure that the Tracked Camera and Lens Component in the image above is set to a specific camera and Lens component. You know it is set up correctly when the text in the image example is white. If there is a problem, yellow text will appear to let you know what is missing. In this example, the Lens component within CineCameraActor1 did not have a Lens file assigned.

Try racking focus on your physical lens before continuing to confirm that you are getting lens data through Live Link. You should see Focus under Raw FIZ Input change in value as you manipulate the lens.

Section 9.19 Mapping Your Camera’s Focus

Select Focus and press the + Button to bring up a new menu titled Add Lens Data Point.

Start by racking focus on your physical lens to its lowest and notice how the value in Input Focus changes. When you’ve confirmed that the lens is at its lowest, enable Input Focus to allow for the editing of Encoder mapping. Enter the value that your lens is reporting for focus distance for this position. Finally, Add to add your first point. My lens is 500cm, which is 500.

Section 9.2 CineCam Lens Component.

Repeat the last step, but this time for the highest focus distance. Technically, you do not need to rack focus on this one as you know it should be full or 100%. In my case, I’ll be using 16300cm.

You should now have a flat line of a graph from 0.0 to 1.0 (or 0 to 100%) focus range of your lowest and highest focus distance. This is, of course, not how a real lens should be, so more points will be needed.

Note that if you rack focus on your physical lens, you should see the Focus scrub bar move on its own from lowest to highest. You should also see a read-out of your lens's current position in Focus Distance, as in this example image below.  

Keep repeating the steps above, adding points from lowest to highest in as many as you can do accurately. Eventually, you’ll have a graph that is similar to the example above. Your Lens file is now complete!

Section 9.21 The Example Lens File.  

Included with the ARFX Pro Plugin is an example Lens file that was used to create the steps above. This lens calibration may not be for your lens, but you can use this as an example of how you should build for your own lens. If you wish to use this file, navigate to where your ARwall plugin folder is in the project within Windows, then drag and drop the file named _ExampleLens.ulens_ into Unreal’s Content Browser.

Section 9.22 Testing Your Lens Emulation.

The quickest way to ensure you are getting lens data to the ARFX Lens Camera is to look into its Details and find the Current Focus in the highlighted areas in the image above. Note that you can only see these values change while in play.

Alternatively, you can turn on the debug focus distance plane to visually see where the focus is currently set to in Unreal. Again, you must be in play for this to work.

Section 9.23 Adjusting your Focus Distance.

You can freely change these modifiers while in play and see the changes in real-time.

Going into your ARFX Lens Camera’s Details panel has several settings within the Lens Emulation section that allow you to fine-tune your Lens Emulation. For example, my lens has a minimum focus distance of 500cm, and I’d rather that the backdrop be fully blurred out by then. So I’ve set Focus Distance Offset to _-500._ Alternatively, you can use Focus Distance Multiplier to expand or contract the calculated focus distance to your preferences.

Section 9.24 Alternatives to Using Lens Component

These options are only useful when not using the Camera Calibrator toolset, which will convert the raw 0 to 100% values from Live Link and FreeD to actual distances.

If you do not wish to use the Live Link Lens component or the Camera Calibrator, you could simply use the raw values coming in from Live Link or Free-D and use Focus Distance Range or Focus Distance Curve instead. You’d still have to manually set up your range or curve, however, so you are not exactly saving any time in doing so. This is simply a complete alternative from established virtual production tools (and I totally didn’t make this before learning about the Camera Calibration toolset…).