Virtual Reality

Contents
Related Links

Overview

Virtual Reality supports the following VR interfaces:

Using Virtual Reality in Prepar3D

  1. Ensure that the headset is plugged in and working, and that the latest runtime and firmware updates for your hardware are installed.
    • For Oculus Rift, ensure that Allow 3rd Party Content is checked in the Oculus Home settings.
  2. Start Prepar3D.
  3. Click on the Virtual Reality menu.
  4. Select Enable Virtual Reality and then select the desired runtime for the headset (SteamVR, Oculus, Varjo, VRG, or HMD Emulator).
  5. Prepar3D should now be visible inside the virtual reality headset.
  6. Select Recalibrate Origin to reset the position of the headset to where you are sitting.
  7. The active view can be changed through the Views menu.

Display Modes

After enabling virtual reality, the menu dropdown will have additional options allowing you to further configure your experience.

Recalibrate Origin

Headset Display Modes

On-Screen Display Modes

Virtual Reality Settings

Configuration options can be found the VirtualReality.xml file located in %PROGRAMDATA%\Lockheed Martin\Prepar3D v5


HeadsetDisplayMode

represents the in headset display mode.

OnScreenDisplayMode

Represents the on screen display which determines how the VR scene is displayed on the external display. There is a small performance advantage to using the Single mode.

SyncMode

This determines when the sync for an HMD will be called. This setting currently only impacts SteamVR and Varjo.

Enable Hidden Area Mesh

Disabling this will remove the hidden area mesh surrounding the VR headset. Default: Enabled

Enable Mirror Texture

Use a mirror texture provided by the HMD for the on screen display. The On Screen Display Mode determines the mirror texture layout. Currently this feature is only supported for Varjo HMD. Enable this to see the image overlay from the Varjo XR camera within the P3D application window. There is a performance cost to using this feature, so it is recommended that this is disabled unless it is required. Default: Disabled

Enable Zoom

Enables the user to zoom in the view while in VR. Default: Disabled

Resolution Scale

Scale resolution of the VR views. Currently only Varjo interface impliments this setting. Most VR platforms have dedicated settings for this. Default: 1.0

Max View Count

Some HMDs can utilize 4 views. As a performance trade off, these HMDs ususally support running with two higher resolution views. Two high resolution views combined with variable rate shading (supported by Varjo and VRG) may perform better than 4. Four lower resolution views can be used when foveated rendering is enabled. This replaces the Varjo Views option which was previously in Varjo specific settings. Default: 4

Eye Tracking Settings

The cursor in Virtual Reality is visualized by a sphere in 3D space based on the mouse position relative to the Prepar3D window. This cursor will not work if you move your mouse to a monitor that is not displaying Prepar3D. The Mouse Cursor and the Virtual Reality cursor will not be at the same location on the screen.

EnableEyeTracking

Turning this on sets the cursor to the eye position. If Eye tracking is not supported the cursor will be placed at the center of the screen. The Vive Pro Eye is currently the only native eye tracking HMD supported by Prepar3D.

EnableGazeDetection

Turning this on displays Green Spheres for Eye Tracking data when P3D is paused, similar to a heat map to show the areas that were looked at most.

EnablePanelEyeData

Turning this on displays a single Green Sphere for where the User is looking along with text data about the Eye such as Openness, Pupil Diameter, and screen Location

EnableGazeSelection

Turning this on locks the cursor to the center of the screen, requiring head movement to move the cursor around.


When Gaze Selection is enabled, you can use the following Key Controls to emulate the functions of mouse clicks to interact with the Virtual Cockpit in VR. This can be useful for users that need mouse controls for interaction with the Virtual Cockpit while wearing a VR Headset, allowing them to 'click' VC elements using keys or buttons on a control surface, instead of operating a mouse. The emulated mouse clicks will interact with the Virtual Cockpit elements underneath the cursor.

To emulate:

GazeDragVelocity

Specifies the speed and direction of the mouse drag when EnableGazeSelection is toggled on. This may need to be adjusted per aircraft.

GazeSelectionOffset

Moves the cursor position (in pixels).

EnableFoveatedRendering

A performance optimization where the image is rendered in higher detail in the area where the eyese are looking. This requires Max View Count be set to Four Views and is currently supported only by the Varjo and VRG interfaces.

EnableVariableRateShading

A performance optimization where the image is rendered in higher detail in the area where the eyese are looking. Instead of using 4 views, this method uses 2 high resolution views. The GPU scales the pixel workload in the focus areas utilzing a Dx12 feature called Variable Rate Shading or VRS. This requires a GPU that supports VRR, and is currently only implimented for Varjo and VRG HMDs.

User Interface Settings

MouseHitDistance

Specifies the max distance (in meters) the object can be for the mouse to detect it. This may need to be adjusted per aircraft..

MouseCursorRadius

Changes the radius of the Virtual Reality Cursor. (Default is 0.0125)

MouseCursorOpacity

Changes the opacity of the Virtual Reality Cursor. (Default is 200)

MouseCursorColor

Changes the color of the Virtual Reality Cursor. (Default is 200, 200, 250)

ShowAppMenu

Enable application menu to display in a 3D window in VR

ShowInfoGen

Enable Information window to display in a 3D window in VR

ShowPortalEditControls

Show portal edit controls help text in a floating 3d window. This will still only display if a passthough camera is enabled and a motion controller is used.

Prepar3D Config


In the Prepar3D.cfg, there are the following Virtual Reality entries:


To automatically enable a VR Interface at Startup, edit the 'EnableVirtualRealityOnStartup' variable found in the Prepar3D.cfg file.

Options for Startup VR Interfaces are:

For Example, to automatically start SteamVR for a VIVE Pro HMD (or any other openVR HMD) when P3D starts, my Prepar3D.cfg file would have the following entry:

EnableVirtualRealityOnStartup=1

Instrument Panels

If an Instrument Panel is opened before entering Virtual Reality, it will be shown in the headset. An Instrument Panel can be opened by selecting the Vehicle menu item and choosing Instrument Panel. Note that not all vehicles have Instrument Panels.


Controls

NOTE: The Virtual Reality cursor must be hovering over the panel to move it.

Developer Mode

Turning on Developer Mode will enable a new headset interface selection in the Virtual Reality drop down menu called HMD Emulator. The HMD Emulator allows Virtual Reality to be enabled without plugging in a headset. Panning can be used to simulate head movement.


Augmented Reality

Overview

Augmented Reality is available for those who have an headset with passthrough camera capabilities (ie. HTC Vive, Varjo, etc.) Developer mode must be enabled to see the Passthrough Camera menu item under Virtual Reality. A passthrough camera must be selected to utilize any AR functionality. The following settings can be found in the AR Settings UI.


EyepointZOffset

Value that corrects the position of camera where the camera is offset from the headset eyepoint.

EnableGreenScreen

Value that enables the green screen for Passthrough cameras. See advanced settings for how to configure chromakey color values.

Passthrough Mode

Selects which mode the passthrough camera will operate in.

Mixed Reality Portals

Portals are masking areas where either the passthrough camera feed is displayed or the Prepar3D scene is displayed. You can enable and disable groups of portals (portal sets) through the drop down menu. An entire set can be deleted in the VR Settings UI. VIVE controllers are necessary for real-time portal placement. Otherwise, they can be created in the UI, or through the Portals.xml file.

Note: You will need to have a Passthrough Camera enabled and your Passthrough Mode set to Portals in order to see portals.

Using Portals

Your controllers will need to be in Portal Edit Mode to directly manipulate portals in the Virtual Environment. Edit mode is enabled if you see the controllers pulsating a green hue. The selected portal will also appear green in this mode. You can select a portal by passing the controller through it. Delete will only delete a selected portal. Creating a portal without having a portal selected will create a new portal set. See the Portal Controls table below for more info on how to enter Portal Edit Mode and the available controls. Delete will only delete a selected portal. Creating a portal without having a portal selected will create a new portal set.

Portal Controls

The first controller that the system recognizes is assigned the Left Hand role. If it feels more natural to have the roles swapped, simply switch the controller hands.


The following table displays the controls for each operating mode. The default mode is entered when an AR pass through camera is selected. Once controllers are connected, you will see 3D representations of the controllers in the Virtual Environment. Pressing the trigger will bring up the Help Text, which hovers over the mouse cursor and displays the controls for the current mode.


Controller Modes when using Mixed Reality Portals:

If you only have one controller, you can use the Grip Button on the Left Controller to emulate Right Controller functions. Simply press the grip button, and the color of the label on the Left Controller will toggle to indicate the controller has switched functionality.


Left Controller Default Mode Portal Mode FOV Mode
DPAD Up/Down Toggle Help Text Create Portal Scale Vertical FOV
DPAD Left/Right Toggle Help Text Cycle Portal Shape Scale Horizontal FOV
Menu Enter FOV Mode Delete Portal Exit FOV Mode
Trigger Toggle Help Text Move Portal ---
Grip Toggle 'Right' Controls Toggle 'Right' Controls Toggle 'Right' Controls
Right Controller Default Mode Portal Mode FOV Mode
DPAD Up/Down Enter Portal Mode Exit Portal Mode Scale Vertical FOV
DPAD Left/Right Enter Portal Mode Cycle Portal Origin Scale Horizontal FOV
Menu Enter FOV Mode --- Exit FOV Mode
Trigger Toggle Help Text Move Portal ---

Eye Tracking

Overview

In Prepar3D, eye tracking takes on a new meaning in training and simulation. Demonstrate your awareness of start-up procedures while controlling knobs and switches as you use your eyes to follow instructions. Review which panels, knobs, and other areas in the current view have been given the most attention by pausing the simlation and seeing the most frequented regions marked by concentrated green circles. Where your eyes travel and how long they remain there provide not only useful functionality but interesting insights that can revolutionize the way you train.

Eye Tracking Setup

Prepar3D currently supports eye tracking using the HTC VIVE Pro Eye, as well as Varjo, and VRG headsets. Because of the uniqueness of these devices, it is important to understand the proper way to enable and configure eye tracking for use in Prepar3D.

Using Eye Tracking with HTC VIVE Pro Eye

HTC VIVE Pro Eye makes use of the SR (Super Reality) runtime to enable eye tracking capabilities. Please ensure the latest version of the SR runtime is installed on your device. This should be installed from having run the VIVE software installer, but may have not been included in the installation if the installer was ran for a prior VIVE (non-VIVE Pro Eye) headset. If you are missing the SR runtime, you may get it by following the instructions at https://www.vive.com/us/support/vive-pro-eye/category_howto/installing-eye-tracking-software.html.

Additional things to note while using eye tracking with the VIVE Pro Eye are the following:

  • The SR runtime will be brought up automatically when eye tracking is enabled and a scenario is loaded in Prepar3D
  • Eye tracking calibration must be done manually by using a connected VIVE remote to enter the VIVE menu, select VIVE Pro Eye, and begin the calibration process
  • In Prepar3D.cfg underneath the "STARTUP" section, "EnableViveProEyeTracking" should be set to 1 to enable VIVE Pro Eye eye tracking - this is the default and may remain enabled unless you desire otherwise
  • Eye tracking calibration with the VIVE Pro Eye is persistent - you may exit Prepar3D and load it again, and/or remove the headset, and the calibration previously made will remain
  • SR_Runtime remains running after exiting both Prepar3D and SteamVR


    • Note: To configure Prepar3D to start up with virtual reality enabled for the HTC VIVE Pro Eye, set "EnableVirtualRealityOnStatup" to 1 in Prepar3D.cfg.

    Using Eye Tracking with Varjo VR-1

    Varjo VR-1 uses Varjo Base to enable both position and eye tracking. Please ensure the latest version of Varjo Base is installed on your device. If you have not yet installed Varjo Base, you may find it at https://varjo.com/downloads/#varjo-base.

    Additional things to note while using eye tracking with the VR-1 are the following:

  • Varjo Base needs to be brought up manually before launching Prepar3D
  • Varjo eye tracking calibration will be brought up automatically when eye tracking is enabled and a scenario is loaded in Prepar3D
  • Once the headset is taken off, even partially, the headset will lose its eye tracking calibration and you will need to calibrate again
  • You may manually start eye tracking calibration via the Virtual Reality menu in Prepar3D


    • Note: To configure Prepar3D to start up with virtual reality enabled for the Varjo VR-1, set "EnableVirtualRealityOnStatup" to 3 in Prepar3D.cfg.

    Advanced Settings (Professional Plus)

    Overview

    These are settings intened for advance mixed reality training use cases where additional tracking capabilities and scene blending techniques are required to correlate objects in the real world with the virtual scene.

    Recalibration Origin

    HMDReferenceOrigin

    Controls the origin Prepar3D will use during recalibration. This is used to set whether Prepar3D will use the zero pose coming in from HMD software as the calibration center, or if Prepar3D will calculate it's own offsets to center the HMD.

    Tracked Device Class

    The class of tracked device to use as a reference for saving and loading the origin offset.

    AutoRestoreOrigin

    If HMDReferenceOrigin is set to P3D Tracker Origin and this option is enabled, the custom tracker origin will be reset shortly after enabling VR. Tracker objects are sorted from right to left relative to the HMD, so it is important if this option is used, that the HMD be generally facing in the correct direction at the time VR is enabled, or the wrong tracker may be selected.

    UseTrackerAsCameraOrigin

    This option will use the trackers location as the camera origin. Each frame the tracker location is updated and a new custom origin is set at the trackers location. This is intended for special cases such as motion platforms. The camera in the training scenario should be offset to be placed at the same location as the tracker in the real world cockpit. This setting is experimental.

    Using Tracker Origin

    Tracker Options: Menu item group that shows up in the Virtual Reality Menu when in P3D Tracker Origin mode

    Before saving or restoring tracker offsets, the tracker system should be calibrated. On calibrate, any custom/tracker origin will be disabled so that calibration behaves the same in tracked origin mode as it does in the default VR zero origin mode. For fine tuning in XR, you may want to hold down the calibrate button/key and move the headset until things look aligned. Then release they key/button. Once calibration is in a good state, chose Tracker Options - Save Offset to save the current tracker offset. If the tracking system experience drift, or the tracking system is calibrated again, the saved origin can be restored using the Tracker Options - Restore Offset. Ideally the fine tuned calibration and saving process only needs to be one once. On subsequent runs, Recalibrate Origin and Restore Offset can be used to get back into the saved origin state without needing the HMD to be in a precise location. If AutoRestoreOrigin is enabled, the custom origin will automatically be restored shortly after enabling VR.


    Note: P3D Custom Origin and Tracker Origin currently only supported for OpenVR and Varjo HMDs.

    The recommended combination of settings to use for XR training with Varjo HMDs is: This configurations used the existing base stations as the tracker reference so that no additional trackers are required. It will also automatically restore the saved offset when VR is enabled.

    KeyColorValue

    Color that will be keyed out and replaced with Prepar3D scene.

    KeyColorThreshold

    How much margin of error is allowable for the key color value. X represents min, Y represents max. The threshold is applied to each color channel (red, green, and blue.) The larger the threshold, the wider the color range being keyed out will be.

    Color Picker

    Sets the KeyColorValue based on the color in the center of the camera's vision. To use, point the camera so that the desired color to be keyed out is roughly in the center of the screen, then press the Color Picker button. The KeyColorValue field will be filled out for you.

    HMD Specific Settings (Professional Plus)

    Varjo Settings and Configuration

    Settings

    VarjoEnableChromaKey

    Enables the XR-1 camera passthrough to overlay the chosen greenscreen color. Disabled by default. (Enable XR1 Camera must be enbaled)

    Enable XR-1/XR-3 Camera

    This will allow the XR camera to be selectable from the camera passthrough system.

    Varjo Chroma Key Color

    This is the chosen color for the chroma key. The default color used below is a pure green RGB of (0, 255, 0). Hue Range: 0 - 360 Saturation and Brightness Range: 0-100, Default: (120, 100, 100)

    Varjo Chroma Key Falloff

    Falloff is the size of the fade at the edge of the tolerance, think of this as a slight blur on the edges of chroma. Typically want to keep these values small. Range: 0-100, Default: (3, 3, 3)

    Varjo Chroma Key Tolerance

    This is the hard cut off of what is considered chroma and what isn't. Range: 0-100, Default: (3, 75, 50)

    Enabling XR-1 Camera

    1. Start Prepar3D.
    2. Activate Varjo (Virtual Reality > Enable Virtual Reality > Varjo)
    3. Open VR Settings (Virtual Reality > VR Setting)
    4. Under Varjo Settings select Yes for Enable XR1 Camera
    5. Activate the XR-1 (Virtual Reality > Passthrough Cameras > XR-1)

    Using Cockpit Masking

    1. Start Prepar3D.
    2. Navigate to Custom Camera Management. (Views > View Management > Manage Cameras)
    3. In the Available Cameras menu choose Virtual Cockpit. (This is selected by default)
    4. In the Camera Effects menu scroll down and enable "MaskVCAlphaAndColor"
    5. In the Camera View menu give your new camera a name. I'll call it "XR-1 View"
    6. (Optional) In the Save To dropdown, choose Global so you won't have to re-create the camera next time you run.
    7. Save the camera and exit out of the Custom Camera Management menu
    8. Change views to the new camera (Views > Change View > Cockpit > XR-1 View