Virtual Reality

Contents
Related Links

Overview

Virtual Reality can be used with the Oculus Rift, HTC Vive, other headsets which support SteamVR, and Varjo VR-1.

Using Virtual Reality in Prepar3D

  1. Ensure that the headset is plugged in and working, and that the latest runtime and firmware updates for your hardware are installed.
    • For Oculus Rift, ensure that Allow 3rd Party Content is checked in the Oculus Home settings.
  2. Start Prepar3D.
  3. Click on the Virtual Reality menu.
  4. Select Enable Virtual Reality and then select the desired runtime for the headset (SteamVR, Oculus, Varjo VR-1, or HMD Emulator).
  5. Prepar3D should now be visible inside the virtual reality headset.
  6. Select Recalibrate Origin to reset the position of the headset to where you are sitting.
  7. The active view can be changed through the Views menu.

Display Modes

After enabling virtual reality, the menu dropdown will have additional options allowing you to further configure your experience.

Recalibrate Origin

Headset Display Modes

On-Screen Display Modes

Virtual Reality Settings

Configuration options can be found the VirtualReality.xml file located in %PROGRAMDATA%\Lockheed Martin\Prepar3D v5


HeadsetDisplayMode

represents the in headset display mode.

OnScreenDisplayMode

represents the on screen display mode.

SyncMode

This determines when the sync for an HMD will be called. This setting currently only impacts SteamVR and Varjo.

Enable Hidden Area Mesh

Disabling this will remove the hidden area mesh surrounding the VR headset. Default: Enabled

Enable Zoom

Enables the user to zoom in the view while in VR. Default: Disabled

Eye Tracking Settings

The cursor in Virtual Reality is visualized by a sphere in 3D space based on the mouse position relative to the Prepar3D window. This cursor will not work if you move your mouse to a monitor that is not displaying Prepar3D. The Mouse Cursor and the Virtual Reality cursor will not be at the same location on the screen.

EnableEyeTracking

Turning this on sets the cursor to the eye position. If Eye tracking is not supported the cursor will be placed at the center of the screen. The Vive Pro Eye is currently the only native eye tracking HMD supported by Prepar3D.

EnableGazeDetection

Turning this on displays Green Spheres for Eye Tracking data when P3D is paused, similar to a heat map to show the areas that were looked at most.

EnablePanelEyeData

Turning this on displays a single Green Sphere for where the User is looking along with text data about the Eye such as Openness, Pupil Diameter, and screen Location

EnableGazeSelection

Turning this on locks the cursor to the center of the screen, requiring head movement to move the cursor around.


When Gaze Selection is enabled, you can use the following Key Controls to emulate the functions of mouse clicks to interact with the Virtual Cockpit in VR. This can be useful for users that need mouse controls for interaction with the Virtual Cockpit while wearing a VR Headset, allowing them to 'click' VC elements using keys or buttons on a control surface, instead of operating a mouse. The emulated mouse clicks will interact with the Virtual Cockpit elements underneath the cursor.

To emulate:

GazeDragVelocity

Specifies the speed and direction of the mouse drag when EnableGazeSelection is toggled on. This may need to be adjusted per aircraft.

GazeSelectionOffset

Moves the cursor position (in pixels).

Cursor Settings

MouseHitDistance

Specifies the max distance (in meters) the object can be for the mouse to detect it. This may need to be adjusted per aircraft..

MouseCursorRadius

Changes the radius of the Virtual Reality Cursor. (Default is 0.0125)

MouseCursorOpacity

Changes the opacity of the Virtual Reality Cursor. (Default is 200)

MouseCursorColor

Changes the color of the Virtual Reality Cursor. (Default is 200, 200, 250)

Prepar3D Config


In the Prepar3D.cfg, there are the following Virtual Reality entries:


To automatically enable a VR Interface at Startup, edit the 'EnableVirtualRealityOnStartup' variable found in the Prepar3D.cfg file.

Options for Startup VR Interfaces are:

For Example, to automatically start SteamVR for a VIVE Pro HMD (or any other openVR HMD) when P3D starts, my Prepar3D.cfg file would have the following entry:

EnableVirtualRealityOnStartup=1

Instrument Panels

If an Instrument Panel is opened before entering Virtual Reality, it will be shown in the headset. An Instrument Panel can be opened by selecting the Vehicle menu item and choosing Instrument Panel. Note that not all vehicles have Instrument Panels.


Controls

NOTE: The Virtual Reality cursor must be hovering over the panel to move it.

Developer Mode

Turning on Developer Mode will enable a new headset interface selection in the Virtual Reality drop down menu called HMD Emulator. The HMD Emulator allows Virtual Reality to be enabled without plugging in a headset. Panning can be used to simulate head movement.


Augmented Reality

Overview

Augmented Reality is available for those who have an headset with passthrough camera capabilities (ie. HTC Vive, Varjo, etc.) Developer mode must be enabled to see the Passthrough Camera menu item under Virtual Reality. A passthrough camera must be selected to utilize any AR functionality. The following settings can be found in the AR Settings UI.


EyepointZOffset

Value that corrects the position of camera where the camera is offset from the headset eyepoint.

EnableGreenScreen

Value that enables the green screen for Passthrough cameras. See advanced settings for how to configure chromakey color values.

Passthrough Mode

Selects which mode the passthrough camera will operate in.

Mixed Reality Portals

Portals are masking areas where either the passthrough camera feed is displayed or the Prepar3D scene is displayed. You can enable and disable groups of portals (portal sets) through the drop down menu. An entire set can be deleted in the VR Settings UI. VIVE controllers are necessary for real-time portal placement. Otherwise, they can be created in the UI, or through the Portals.xml file.

Note: You will need to have a Passthrough Camera enabled and your Passthrough Mode set to Portals in order to see portals.

Using Portals

Your controllers will need to be in Portal Edit Mode to directly manipulate portals in the Virtual Environment. Edit mode is enabled if you see the controllers pulsating a green hue. The selected portal will also appear green in this mode. You can select a portal by passing the controller through it. Delete will only delete a selected portal. Creating a portal without having a portal selected will create a new portal set. See the Portal Controls table below for more info on how to enter Portal Edit Mode and the available controls. Delete will only delete a selected portal. Creating a portal without having a portal selected will create a new portal set.

Portal Controls

The first controller that the system recognizes is assigned the Left Hand role. If it feels more natural to have the roles swapped, simply switch the controller hands.


The following table displays the controls for each operating mode. The default mode is entered when an AR pass through camera is selected. Once controllers are connected, you will see 3D representations of the controllers in the Virtual Environment. Pressing the trigger will bring up the Help Text, which hovers over the mouse cursor and displays the controls for the current mode.


Controller Modes when using Mixed Reality Portals:

If you only have one controller, you can use the Grip Button on the Left Controller to emulate Right Controller functions. Simply press the grip button, and the color of the label on the Left Controller will toggle to indicate the controller has switched functionality.


Left Controller Default Mode Portal Mode FOV Mode
DPAD Up/Down Toggle Help Text Create Portal Scale Vertical FOV
DPAD Left/Right Toggle Help Text Cycle Portal Shape Scale Horizontal FOV
Menu Enter FOV Mode Delete Portal Exit FOV Mode
Trigger Toggle Help Text Move Portal ---
Grip Toggle 'Right' Controls Toggle 'Right' Controls Toggle 'Right' Controls
Right Controller Default Mode Portal Mode FOV Mode
DPAD Up/Down Enter Portal Mode Exit Portal Mode Scale Vertical FOV
DPAD Left/Right Enter Portal Mode Exit Portal Mode Scale Horizontal FOV
Menu Enter FOV Mode --- Exit FOV Mode
Trigger Toggle Help Text Move Portal ---

Eye Tracking

Overview

In Prepar3D, eye tracking takes on a new meaning in training and simulation. Demonstrate your awareness of start-up procedures while controlling knobs and switches as you use your eyes to follow instructions. Review which panels, knobs, and other areas in the current view have been given the most attention by pausing the simlation and seeing the most frequented regions marked by concentrated green circles. Where your eyes travel and how long they remain there provide not only useful functionality but interesting insights that can revolutionize the way you train.

Eye Tracking Setup

Prepar3D currently supports eye tracking using the HTC VIVE Pro Eye and Varjo VR-1. Because of the uniqueness of these devices, it is important to understand the proper way to enable and configure eye tracking for use in Prepar3D.

Using Eye Tracking with HTC VIVE Pro Eye

HTC VIVE Pro Eye makes use of the SR (Super Reality) runtime to enable eye tracking capabilities. Please ensure the latest version of the SR runtime is installed on your device. This should be installed from having run the VIVE software installer, but may have not been included in the installation if the installer was ran for a prior VIVE (non-VIVE Pro Eye) headset. If you are missing the SR runtime, you may get it by following the instructions at https://www.vive.com/us/support/vive-pro-eye/category_howto/installing-eye-tracking-software.html.

Additional things to note while using eye tracking with the VIVE Pro Eye are the following:

  • The SR runtime will be brought up automatically when eye tracking is enabled and a scenario is loaded in Prepar3D
  • Eye tracking calibration must be done manually by using a connected VIVE remote to enter the VIVE menu, select VIVE Pro Eye, and begin the calibration process
  • In Prepar3D.cfg underneath the "STARTUP" section, "EnableViveProEyeTracking" should be set to 1 to enable VIVE Pro Eye eye tracking - this is the default and may remain enabled unless you desire otherwise
  • Eye tracking calibration with the VIVE Pro Eye is persistent - you may exit Prepar3D and load it again, and/or remove the headset, and the calibration previously made will remain
  • SR_Runtime remains running after exiting both Prepar3D and SteamVR


    • Note: To configure Prepar3D to start up with virtual reality enabled for the HTC VIVE Pro Eye, set "EnableVirtualRealityOnStatup" to 1 in Prepar3D.cfg.

    Using Eye Tracking with Varjo VR-1

    Varjo VR-1 uses Varjo Base to enable both position and eye tracking. Please ensure the latest version of Varjo Base is installed on your device. If you have not yet installed Varjo Base, you may find it at https://varjo.com/downloads/#varjo-base.

    Additional things to note while using eye tracking with the VR-1 are the following:

  • Varjo Base needs to be brought up manually before launching Prepar3D
  • Varjo eye tracking calibration will be brought up automatically when eye tracking is enabled and a scenario is loaded in Prepar3D
  • Once the headset is taken off, even partially, the headset will lose its eye tracking calibration and you will need to calibrate again
  • You may manually start eye tracking calibration via the Virtual Reality menu in Prepar3D


    • Note: To configure Prepar3D to start up with virtual reality enabled for the Varjo VR-1, set "EnableVirtualRealityOnStatup" to 3 in Prepar3D.cfg.

    Advanced Settings (Professional Plus)

    Overview

    These are settings intened for advance mixed reality training use cases where additional tracking capabilities and scene blending techniques are required to correlate objects in the real world with the virtual scene.

    Recalibration Origin

    HMDReferenceOrigin Controls the origin Prepar3D will use during recalibration. This is used to set whether Prepar3D will use the zero pose coming in from HMD software as the calibration center, or if Prepar3D will calculate it's own offsets to center the HMD.

    Tracker Origin

    To configure a VIVE tracker puck or controller as the reference origin, connect the tracked object through Steam VR, and select 'P3D Tracker Origin' as the HMDReferenceOrigin from the VR settings menu. Once selected, place the tracked object in its desired location within view of the lighthouses, and hold the HMD where you would like to save as the origin. Select 'Save Tracker Offset' from the Virtual Reality drop down menu, and then select 'Recalibrate Origin'. Moving the tracked object and recalibrating again will move the headset so it maintains the previously saved offset, now based from the tracked object's new position.


    Note: P3D Custom Origin and Tracker Origin currently only supported for HMDs using openVR.

    KeyColorValue

    Color that will be keyed out and replaced with Prepar3D scene.

    KeyColorThreshold

    How much margin of error is allowable for the key color value. X represents min, Y represents max. The threshold is applied to each color channel (red, green, and blue.) The larger the threshold, the wider the color range being keyed out will be.

    Color Picker

    Sets the KeyColorValue based on the color in the center of the camera's vision. To use, point the camera so that the desired color to be keyed out is roughly in the center of the screen, then press the Color Picker button. The KeyColorValue field will be filled out for you.

    HMD Specific Settings (Professional Plus)

    Varjo Settings and Configuration

    Settings

    VarjoEnableChromaKey

    Enables the XR-1 camera passthrough to overlay the chosen greenscreen color. Disabled by default. (Enable XR1 Camera must be enbaled)

    Enable XR-1 Camera

    This will allow the XR-1's camera system to be selectable from the camera passthrough system.

    Varjo Chroma Key Color

    This is the chosen color for the chroma key. The default color used below is a pure green RGB of (0, 255, 0). Hue Range: 0 - 360 Saturation and Brightness Range: 0-100, Default: (120, 100, 100)

    Varjo Chroma Key Falloff

    Falloff is the size of the fade at the edge of the tolerance, think of this as a slight blur on the edges of chroma. Typically want to keep these values small. Range: 0-100, Default: (3, 3, 3)

    Varjo Chroma Key Tolerance

    This is the hard cut off of what is considered chroma and what isn't. Range: 0-100, Default: (3, 75, 50)

    Varjo Views

    This setting allows users to switch between rendering either two views or four views

    Enabling XR-1 Camera

    1. Start Prepar3D.
    2. Activate Varjo (Virtual Reality > Enable Virtual Reality > Varjo)
    3. Open VR Settings (Virtual Reality > VR Setting)
    4. Under Varjo Settings select Yes for Enable XR1 Camera
    5. Activate the XR-1 (Virtual Reality > Passthrough Cameras > XR-1)

    Using Cockpit Masking

    1. Start Prepar3D.
    2. Navigate to Custom Camera Management. (Views > View Management > Manage Cameras)
    3. In the Available Cameras menu choose Virtual Cockpit. (This is selected by default)
    4. In the Camera Effects menu scroll down and enable "MaskVCAlphaAndColor"
    5. In the Camera View menu give your new camera a name. I'll call it "XR-1 View"
    6. (Optional) In the Save To dropdown, choose Global so you won't have to re-create the camera next time you run.
    7. Save the camera and exit out of the Custom Camera Management menu
    8. Change views to the new camera (Views > Change View > Cockpit > XR-1 View