Oculus Unreal software User manual

Testing and Performance Analysis in Unreal

on page 5.

This guide assumes you have installed Visual Studio 2015 and are familiar with its use.

Several of these steps may take some time to complete - some of them typically take over an hour, depending on your computer and the speed of your Internet connection.

Note: Visual Studio 2013 is no longer supported.

1. Clone or download the UE4 source code from the Oculus GitHub repository here:

https://github.com/

Oculus-VR/UnrealEngine

. To access this repository, you must hold a UE4 license and have access to the private Epic GitHub repository (see

https://www.unrealengine.com/ue4-on-github

for details).

2. If you downloaded a zip archive of the code, extract the archive contents to a directory where you would like to build the engine, e.g., C:\Unreal\4.13-oculus. We recommend installing to a short path, or you may have errors in the next step with files that exceed the Windows length limit for file names. Alternately, you can map your install directory as a Windows drive to reduce the path length.

3. Run Setup.bat.

4. Run GenerateProjectFiles.bat.

5. Launch UE4.sln to open the project solution in Visual Studio 2015.

6. In the menu bar, select

Build > Configuration Manager. Verify that Active solution configuration is set to

Development Editor, and that Active solution platform is set to Win64.

7. In the Solution Explorer, right-click

UE4 under Engine and select Set as Startup Project. Then select Build in the same context menu.

8. To launch the engine:

• With command-line arguments: Right-click UE4 in the Solution Explorer and select

Properties. In the

UE Property Pages dialog, select

Configuration Properties > Debugging on the left. Enter any desired configuration options in the

Command Arguments field on the top.

• Without command-line arguments: In the Solution Explorer on the right, right-click UE4 under Engine and, in the context menu, select

Debug > Start New Instance to launch the engine.

10 | Visual Studio Quick Start | Unreal

9. Select the project you would like to open, or specify a new project. If you are creating a new project, don’t forget to specify a project name.

10.At this point, the engine will close and a new instance of Visual Studio will launch with your selected or new project. Repeat step 6 to launch the engine with the specified project.

For Epic's instructions on building the Unreal Engine from source, see

Building Unreal Engine from Source

guide.

Unreal | Getting Started | 11

Getting Started

This guide reviews the basics for beginning Oculus development in Unreal.

Overview

The easiest way to get started with Oculus development in Unreal is to experiment with the HMD support provided by default as a Player Start. If Unreal detects an Oculus Rift runtime (e.g., the Oculus app is installed), it will automatically provide orientation and positional tracking and stereoscopic rendering for any game.

Standalone PC executables will automatically display in Oculus Rift when run in full-screen mode if a Rift is detected on your system. You may build a project normally and preview it in the Engine by selecting the VR

Preview mode in the Play button pulldown options.

Note: In order to play an in-development application, you will need to enable running applications from unknown sources in the Oculus app settings, available through the gear icon in the upper-right. Select

Settings > General and toggle Unknown Sources on to allow.

The first time you run an application that you have not downloaded from the Oculus Store, you will need to launch it directly. Once you have run an application from an unknown source at least once, it will then appear in the Library section of Home and the Oculus app, and may be launched normally, as long as

Unknown Sources is enabled.

To preview a level in the Rift while editing with the engine, select the dropdown arrow near the Play button and select VR Preview.

12 | Getting Started | Unreal

To play a scene in the Rift with a standalone PC game, simply maximize the game window and it will be mirrored to the Rift display.

Note: While Unreal is running, you will not be able to access Oculus Home by putting on the headset.

Adding a VR Camera

Using the default camera set to VR Preview is a good way to get a quick sense of VR development with minimal overhead, but for actual development, we recommend adding a Camera actor to your map and enabling

Oculus support by selecting the camera in Viewport and checking the

Lock to Hmd checkbox in the Details tab (available in UE4 11).

Unreal | Getting Started | 13

Placing a camera in the scene allows you to control the orientation of the camera view when the game loads, so that you can control the exact perspective that will be visible to the user. This is not possible with the Play Start described above.

An additional benefit to using the Camera actor is that you can attach meshes and they will update their position following the HMD view with very little latency. This is generally the best way to add head-locked elements such as cockpit details. Note that we generally discourage head-locked UI elements, but it can be an attractive feature when used carefully.

HMD Pose Tracking Origin

The HMD pose is reported relative to a tracking origin, which may be set to two values in your

HeadMountedDisplay settings:

Eye Level

Floor Level

The initial pose of the HMD when the camera activates.

The origin set by the user during Rift calibration, typically near the center of their playable area. Unlike eye-level, the floor-level origin's position is on the floor, at Y = 0.

When you recenter the tracking origin, the behavior is different for eye and floor level.

14 | Getting Started | Unreal

Eye level moves the origin's position (X, Y, and Z) and the yaw (Y rotation) to match the current head pose. Pitch and roll (X and Z rotation) are not modified because that would make the app's virtual horizon mismatch the real-world horizon, disorienting the user.

Floor level moves the origin's X and Z position, but leaves the Y position alone to keep it consistent with the real-world floor. Rotation is handled the same way as eye level.

For more information, see:

•

Tracking

in our Intro to VR

•

IHeadMountedDisplay

in the Unreal API Guide

VR Template

Unreal Engine v4.13 and later include a Virtual Reality Blueprint project template which may be selected when creating a New Project.

Unreal | Getting Started | 15

VR Template contains two maps, accessible through the Content Browser in

Content > VirtualRealityBP >

Maps.

The HMD Locomotion Map is a simple level that demonstrates teleportation. Set your travel destination with the blue gaze-controlled circle. Once the teleport target is set, press the spacebar or gamepad button to teleport. You may optionally control the the orientation you will be facing on arrival with the gamepad primary joystick.

The Motion Controller Map also demonstrates teleportation control, in this case using tracked Touch controllers. Point in the direction you’d like to travel with the Touch controller and and control the destination orientation with your gaze. Then press the A-button to teleport.

16 | Getting Started | Unreal

Use the trigger buttons to pick up and manipulate the small blue blocks in the level.

Unreal | Getting Started | 17

18 | Unreal Mobile Development | Unreal

Unreal Mobile Development

This guide covers environment setup, project configuration, and development for the Oculus mobile platform using Unreal.

Overview

Unreal versions 4.10 and later provide built-in support for Oculus mobile development. Projects configured for

VR automatically apply stereoscopic rendering and orientation tracking. Once your environment is set up with the appropriate tools and your project settings are configured properly, you may build virtual reality projects targeting Android, load APKs on your mobile device, and play them on the Oculus Mobile platform.

Note: Mobile applications are subject to more computational limitations compared to Rift applications, and this should be taken into consideration beginning in the early design phase.

Oculus Mobile SDK

Oculus provides a mobile SDK with native C/C++ libraries for Android development in addition to supporting mobile development for game engines such as Unreal. It is not necessary for Unreal developers to download or install the Mobile SDK, but you may wish to look through our

Mobile SDK Developer Guide

for general information on Gear VR development such as application signature, basic mobile development tools such as adb and Oculus Remote Monitor, and performance guidelines.

We recommend that Unreal mobile developers review the following sections:

•

Mobile SDK Getting Started Guide

•

Mobile Development Basics

•

Mobile Best Practices

•

Testing and Troubleshooting

Developers interested in lower-level details about how the mobile VR rendering pipeline is handled by our native libraries may wish to download the mobile SDK and review the headers, particularly for the

VrApi

, which is responsible for returning headset orientation data and applying distortion, sensor fusion, and compositing.

Mobile Environment Setup

There are two ways to set up your environment for Gear VR development for Unreal: 1) using the Codeworks for Android installation package (recommended), or 2) using the standard Android SDK.

Epic strongly recommends using the Codeworks for Android installation package rather than the stock Android

SDK. The Codeworks installer is bundled with the Unreal installation in Engline\Extras\AndroidWorks. Using this package simplifies Unreal set up and is usually necessary to successfully build Gear VR projects. However, it requires uninstalling any standard Android SDK installation and may interfere with development using other engines or IDEs that require the standard Android SDK. To install Codeworks for Android 1R4, follow the instructions in Epic’s

Required Android Setup

guide.

Note: You will find the required version of Codeworks here: Engine\Extras\AndroidWorks\Win64.

To prepare for Gear VR development using the standard Android SDK, follow the instructions in our Mobile

Android Development Software Setup for Windows

guide to install the Java Development Kit (JDK), Native

Development Kit, and Android SDK. Unreal developers do not need to install Android Studio. Our Unreal SDK does not currently support OS X or Linux.

Unreal | Unreal Mobile Development | 19

Once you have installed the required Android tools, follow the setup instructions described in the

Device Setup

guide in our Mobile SDK documentation. In this process you will enable Developer Options on your mobile device and make a few device configuration settings.

Application Signing

Oculus mobile applications must be signed by two different signatures at different stages of development.

An Oculus Signature File (osig) is required for your application to run on Gear VR, and an Android distribution keystore is required for submission to the Oculus Store.

Once you add an osig to the appropriate Unreal directory, it will be added automatically to every APK that you build. You will need one osig for each mobile device.

To add your osig to Unreal for development:

1. Follow the instructions in

Application Signing

to download your osig.

2. Navigate to the directory <Unreal-directory>\Engine\Build\Android\Java\.

3. Create a new directory inside \Engine\Build\Android\Java\ and name it assets. The name must not be capitalized.

4. Copy your osig to this directory.

When you are ready to build an APK for submission to release, we recommend that you exclude the osig in your APK. To do so, select

Edit > Project Settings > Android, scroll down to Advanced APKPackaging, and verify that

Remove Oculus Signature Files from Distribution APK is checked.

20 | Unreal Mobile Development | Unreal

Before building your final release build, create a new Android keystore by following the “Sign Your App

Manually” instructions in Android's

Sign your Applications

guide. Once you have generated your distribution keystore, go to

Edit > Project Settings > Platforms > Android, scroll down to Distribution Signing, and entered the required information.

See the

Application Signing

section of the Mobile SDK documentation for more information.

Configure Unreal for Android SDK

If you do not have a project prepared but would like to try out the process, you may use create a scene with starter content such as the C++ First Person project.

Unreal | Unreal Mobile Development | 21

Once you have installed the Android SDK and required tools, configure Unreal for Android development.

1. Select

Edit > Project Settings.

2. In the

Project Settings menu on the left, go to Platforms and select Android SDK (not Android - we will configure that later).

3. Configure all fields in

SDKConfig with the appropriate paths to the necessary tools. Note: if you installed

Codeworks for Android, these fields should be populated automatically.

Project Configuration

This section describes how to configure any C++ or Blueprints Unreal project targeting desktop or mobile for use with Oculus mobile devices.

If you do not have a project prepared but would like to try out the process, you may use create a scene with starter content such as the C++ First Person project.

1. Select

Edit > Plugins.

2. Select

Virtual Reality.

3. Verify that the

Enabled check box is checked for OculusVR. If you need to select it, click Restart Now in the lower-right afterward. Close Plugins configuration.

4. Select

Edit > Project Settings.

5. Fill in any other desired information in

Project > Description.

6. In the

Project Settings menu on the left, go to Platforms and select Android.

7. Under

APKPackaging, set the Minimum SDK Version to 21, and set the Target SDK Version to 21.

8. Scroll down to

Advanced APKPackaging. Then: a. Check

Configure the AndroidManifest for deployment to GearVR.

b. Verify that

Remove Oculus Signature Files from Distribution APK is unchecked, unless you are building a package to release.

9. In

Engine > Rendering, uncheck Mobile HDR in the Mobile section. Restart project if prompted to do so.

10.Close the

Project Settings configuration window.

Building and Running Projects

1. Connect your Samsung phone to your PC via USB.

2. Open a shell terminal and verify that your device is communicating with your PC using adb devices.

Note that depending on the device you are using, you may need to configure your connection for software installation. For more information, see

Adb

in our Mobile SDK Developer Guide.

3. We recommend using ASTC compression, though ETC2 will also work. Select

File > Package Project >

Android > Android (ASTC).

4. When prompted, browse to the destination folder where you would like your APK to be installed.

5. Once the build process is completed, navigate to the destination folder. Run the .bat file beginning with

Install_ to install the application to your phone.

6. Once the build process is complete, your application will be visible on your mobile devices under

Apps.

Note that it will not be visible in Oculus Home.

7. Click the application to launch. You will be prompted to insert it into your Gear VR headset.

Note: You may also set your phone to

Developer Mode

, which allows you to run VR applications on your mobile device without inserting it into an HMD.

Launching a Project Directly onto your Phone

You may also directly build and launch an application to your phone without saving the APK locally.

22 | Unreal Mobile Development | Unreal

1. Connect your phone to your PC by USB.

2. Select the

Launch menu from the Unreal toolbar and select your phone under Devices. If you do not see your phone listed, verify your USB connection, and check if you need to set your connection to

Connected as an Installer - some Samsung phones default to Charge Only connection.

3. Your application will build and install to your Android device. When the build is complete, you will be prompted to insert the phone into your headset to launch.

Workflow

We recommend setting your phone to

Developer Mode

, so you can run VR apps on your phone without inserting it into the Gear VR headset. This allows you to quickly review your progress on the device without much overhead.

You may also launch the engine with a configuration option to use the GearVR plugin and Mobile renderer on the PC. This allows you to preview mobile development from the desktop using an Oculus Rift. To do so, disable OculusRift and SteamVR plugins for your project, and add

-OpenGL -FeatureLevelES2 to your command line.

To run your app during development, we generally recommend connecting your phone via USB, selecting the Launch pulldown menu, and then selecting your phone under the listed devices. Particularly if you are modifying maps and shaders but not changing the code, this is often much faster than building an APK.

It is possible to preview a Gear VR application in the Oculus Rift during development. However, it is not generally useful to do so, because Rift applications are subject to substantially different performance requirements (see "Performance Targets" in

on page 79). You may find it easiest to use the 2D preview in Unreal, and then either build an APK or use Launch on Device when you need to view the app in VR.

Note: Mobile Content Scale Factor is not currently supported for Oculus development.

Blueprints

We provide Blueprints for common Gear VR operations such as querying battery level and headphone connection status.

For more information, see

Advanced Rendering Features

on page 51.

Advanced Rendering Features for Mobile

The Oculus SDK offers advanced rendering feature for Gear VR such as multi-view and hybrid monoscopic rendering. Under some conditions they can significantly improve performance.

For more information, see

on page 32

Unreal | Unreal Input | 23

Unreal Input

This section describes input handling for Oculus devices.

Gear VR Controller

The Gear VR Controller is an orientation-tracked input device available through Unreal as a Motion Controller.

For a discussion of best practices, see

Gear VR Controller Best Practices

in Oculus Best Practices.

For instructions on how to add a Motion Controller component to your Pawn or Character, see

Motion

Controller Component Setup

in Epic’s Unreal documentation. Unreal has also provided a detailed training tutorial called

Setting Up VR Motion Controllers

Gear VR positions the controller relative to the user by using a body model to estimate the controller’s position.

Whether the controller is visualized on the left or right side of the body is determined by left-handedness versus right-handedness, which is specified by users during controller pairing.

Orientation tracking is handled automatically by the Motion Controller Component. If you need to query the controller orientation, you can query the Motion Controller rotation.

Add the GearVRControllerComponent to create a MotionController with a Gear VR Controller mesh as a child

(4.15 and later). The Gear VR Controller mesh may be found in Plugins/GearVR/Content/.

Motion Controller Components must be specified as either left or right controllers when they are added, and each Gear VR Controller button mapping has a left/right equivalent. However, any button click sends both left and right events, so the setting you choose when add the Motion Controller component has no effect.

Input Sample

You will find an example of Gear VR Controller input in our Input sample available in the directory <install>/

Samples/Oculus. Please see the sample and its Level Blueprint for a full illustration of how to use the controller in your game, including the button mappings.

Gear VR Controller Swiping Gestures

For Gear VR Controllers, the user interface of your VR experience should follow these natural scrolling and swiping gestures:

• Swipe up: Pull content upward. Equivalent to scrolling down.

• Swipe down: Pull content downward. Equivalent to scrolling up.

• Swipe left: Pull content left or go to the next item or page.

• Swipe right: Pull content right or go to the previous item or page.

Haptics for Rift Controllers

This guide describes how to use Unreal Blueprints to control haptic effects on Touch or Xbox controllers.

You may use the standard

Play Haptic Effect Blueprint to send a specified haptic curve to the Oculus Touch or

Xbox controller. For more information, see Unreal’s

Play Haptic Effect guide

24 | Unreal Input | Unreal

Haptics in Unreal Engine 4.13

PlayHapticEffects may be configured to play haptic waves based on three types of input. Right-click Content

Browser to bring up the context menu, then select Miscellaneous. and select one of the following three options:

• Haptic Feedback Buffer: Plays a buffer of bytes 0-255,

• Haptic Feedback Curve: Draw the haptic linear curve you wish to play using the Haptic Curve Editor, or

• Haptic Feedback Soundwave: Select a mono audio file to be converted into a haptic effect of corresponding amplitude.

The following Blueprint illustrates a simple haptics sequence on the Oculus Touch controller using

Play Haptic

Effect. This example sends vibrations using Play Haptic Effect when the left controller grip button is pressed.

When the button is released,

Stop Haptic Effect sends a stop command to the Touch controller.

When the left controller X button is pressed, a constant vibration is sent by

Set Haptics by Value until the button is released. Note that

Set Haptics by Value calls are limited to 30 Hz; additional calls will be disregarded.

Unreal | Unreal Input | 25

26 | Unreal Input | Unreal

Haptics in Unreal Engine 4.12

In addition to Play Haptic Effects, Unreal 4.12 adds Play Haptic Soundwave.

The following Blueprint illustrates a simple haptics sequence on the Oculus Touch controller using

Play Haptic

Effects and Play Haptic Soundwave. This example sends vibrations using Play Haptic Effect when the left controller grip button is pressed. When the button is released,

Play Haptic Soundwave sends a second vibration to the controller.

When the left controller X button is pressed, a constant vibration is sent by

Set Haptics by Value until the button is released. Note that

Set Haptics by Value calls are limited to 30 Hz; additional calls will be disregarded.

Unreal | Unreal Input | 27

28 | Unreal Input | Unreal

APlayerController::PlayHapticSoundWave takes a mono soundwave as an argument. It downsamples the wave into a series of bytes that serially describe the amplitude of the wave (uint8 values 0-255). Each byte is then multiplied by the factor specified in Scale (max = 255), and haptic vibrations are sent to the targeted

Oculus Touch controller. Each controller must be targeted individually. Call

Stop Haptic Effect to stop haptic playback.

Haptics Sample

The TouchSample, available from our Unreal GitHub repository, illustrates basic use of Oculus Touch, including haptics control using

PlayHapticEffect() and PlayHapticSoundWave(). For more information, see

Unreal Samples

on page 71.

Guardian System Boundary Component

OculusBoundaryComponent exposes an API for interacting with the Oculus Guardian System.

During Touch setup, users define an interaction area by drawing a perimeter called the Outer Boundary in space with the controller. An axis-aligned bounding box called the Play Area is calculated from this perimeter.

When tracked devices approach the Outer Boundary, the Oculus runtime automatically provides visual cues to the user demarcating the Outer Boundary. This behavior may not be disabled or superseded by applications, though the Guardian System visualization may be disabled via user configuration in the Oculus app.

Additional handling may be implemented by applications using the class

UOculusBoundaryComponent.

Possible use cases include pausing the game if the user leaves the Play Area, placing geometry in the world

Unreal | Unreal Input | 29

based on boundary points to create a “natural” integrated barrier with in-scene objects, disabling UI when the boundary is being rendered to avoid visual discomfort, et cetera.

All

UOculusBoundaryComponent public methods are available as Blueprints.

Please see OculusBoundaryComponent.h for additional details.

Basic Use

Boundary types are

Boundary_Outer and Boundary_PlayArea.

Device types are

HMD, LTouch, RTouch, Touch (i.e., both controllers), and All.

Applications may query the interaction between devices and the Outer Boundary or Play Area by using UOculusBoundaryComponent::GetTriggeredPlayAreaInfo() or

UOculusBoundaryComponent::GetTriggeredOuterBoundaryInfo().

Applications may also query arbitrary points relative to the Play Area or Outer

Boundary using UOculusBoundaryComponent::CheckIfPointWithinOuterBounds() or

UOculusBoundaryComponent::CheckIfPointWithinPlayArea(). This may be useful for determining the location of particular Actors in a scene relative to boundaries so, for example, they are spawned within reach, et cetera.

Results are returned as a struct called FBoundaryTestResult, which includes the following members:

Type Description Member

IsTriggering

DeviceType

ClosestDistance

ClosestPoint

ClosestPointNormal bool

ETrackedDeviceTypeDevice type triggering boundary.

float Distance between the device or point and the closest point of the test area.

FVector

Returns true if the device or point triggers the queried boundary type.

FVector

Describes the location in tracking space of the closest boundary point to the queried device or point.

Describes the normal of the closest boundary point to the queried device or point.

All dimensions, points, and vectors are returned in Unreal world coordinate space.

Applications may request that boundaries be displayed or hidden using

RequestOuterBoundaryVisible(). Note that the Oculus runtime will override application requests under certain conditions. For example, setting Boundary Area visibility to false will fail if a tracked device is close enough to trigger the boundary’s automatic display. Setting the visibility to true will fail if the user has disabled the visual display of the boundary system.

Applications may query the current state of the boundary system using

UOculusBoundaryComponent::IsOuterBoundaryDisplayed() and

UOculusBoundaryComponent::IsOuterBoundaryTriggered().

You may bind delegates using the object

OnOuterBoundaryTriggered.

30 | Unreal Input | Unreal

Additional Features

You may set the boundary color of the automated Guardian System visualization (alpha is unaffected) using

UOculusBoundaryComponent::SetOuterBoundaryColor(). Use

UOculusBoundaryComponent::ResetOuterBoundaryColor()to reset to default settings.

UOculusBoundaryComponent::GetOuterBoundaryPoints() and

UOculusBoundaryComponent::GetPlayAreaPoints()return an array of up to 256 3D points that define the Boundary Area or Play Area in clockwise order at floor level. You may query the dimensions of a Boundary

Area or Play Area using

UOculusBoundaryComponent::GetOuterBoundaryDimensions() or

UOculusBoundaryComponent::GetPlayAreaDimensions(), which return a vectors containing the width, height, and depth in tracking space units, with height always returning 0.

Boundary Sample

BoundarySample, available from our Unreal GitHub repository, illustrates the use of the Boundary Component

API for interacting with our Guardian System. For more information, see

Unreal Samples

on page 71.

Unreal | Unreal VR Audio | 31

Unreal VR Audio

Audio is critical for creating a persuasive VR experience and can contribute strongly to the user's sense of immersion.

Audio Basics

When Unreal PC applications are launched, if the OculusVR plugin (or OculusRift plugin in legacy versions) is enabled and the Oculus VR Runtime Service is installed, then the application will automatically override the default Windows graphics and audio devices and target the Rift. The Oculus VR Runtime Service is installed with the Oculus app.

Unless your application is intended to run in VR, do not enable the OculusVR plugin. Otherwise, it is possible that audio and/or video will be incorrectly targeted to the Oculus Rift when the application is run.

Alternatively, users can disable loading all HMD plugins by specifying "-nohmd" on the command line.

Audio Spatialization

The Oculus Audio SDK includes spatialization plugins (OSPs) that provide HRTF spatialization and reverb modeling for audio editing tools commonly used with Unreal, including Audiokinetic Wwise and FMOD Studio.

FMOD supports both Rift and mobile development, and Wwise supports Rift development. For more details on integrating our spatialization plugins with FMOD and Wwise for use in Unreal, see the

Rift Audio

section of our

Rift developer guide.

We recommend using FMOD or Wwise with the appropriate OSP, which will provide access to our full spatialization feature set as well as the full functionality of the audio tools themselves. Epic also offers built-in audio spatialization for Rift with HRTF-spatialization only. Epic’s 4.15-preview branch offers preview support for

Gear VR which does not support some features such as ducking and filtering, and which may contain bugs. For an example of how to implement it, see AmbientSound Spatialize in the Unreal

Audio Content Example

For more information on our plugins, and for general information about audio design for VR, please see our

Audio SDK developer guide

32 | Advanced Rendering Features | Unreal

Advanced Rendering Features

This section describes important rendering options and tools that can significantly improve your application.

Unreal Forward Shading Renderer (PC)

Unreal Engine 4.14 introduced an experimental forward shading renderer optimized for VR. We recommend that all future PC titles use the forward shading renderer.

The primary advantage of the forward renderer is that it is substantially faster than the deferred renderer. It outputs lit pixels directly, whereas the deferred renderer outputs material properties to a set of buffers, and then reads them back to apply lighting in a separate pass.

Because the forward renderer does not render to intermediate buffers, it can take advantage of MSAA. We recommend using MSAA for anti-aliasing, as it not only reduces motion blur, but often provides a significant savings in GPU utilization over TAA.

Not all features from the deferred renderer are available in the forward renderer, but many of these features require tradeoffs that disproportionately impact VR development. For example, space screen effects may introduce stereo disparities that can be uncomfortable for users.

Given the substantial advantages the forward shading renderer offers VR developers, we anticipate that the forward shading renderer will become the target for future VR optimizations.

For more information on the forward shading renderer, see Epic’s

New: Forward Shading Renderer with MSAA

VR Compositor Layers

In some versions of Unreal, you may add transparent or opaque quadrilateral, cubemap, or cylindrical overlays to your level as compositor layers.

TimeWarp compositor layers are rendered at the same frame rate as the compositor, rather than rendering at the application frame rate. They are less prone to judder, and are raytraced through the lenses, which improves the clarity of textures displayed on them.

We recommend using compositor layers for text and for headlocked elements. Text rendered on compositor layers is more legible, and headlocked layers remain headlocked when Asynchronous TimeWarp interpolates dropped frames.

Gaze cursors and UIs are good candidates for rendering as quadrilateral compositor layers. Cylinders may be useful for smooth-curve UI interfaces. Cubemaps may be used for startup scenes or skyboxes.

We recommend using a cubemap compositor layer for your loading scene, so it will always display at a steady minimum frame rate, even if the application performs no updates whatsoever. This can significantly improve application startup times.

Quadrilateral, cylinder, and cubemap layers are supported in Unreal versions 4.13 and later.

By default, VR compositor layers are always displayed on top of all other objects in the scene. You may set compositor layers to respect depth positioning by enabling

Supports Depth. If you are using multiple layers, use the

Priority setting to control the depth order in which the layers will be displayed, with lower values indicating greater priority (e.g., 0 is before 1).

Unreal | Advanced Rendering Features | 33

Note that enabling

Supports Depth may affect performance, so use it with caution and be sure to assess its impact.

To create an overlay:

1. Create an Actor and add it to the level. If you don’t want the Actor to be visible in the scene, use an Empty

Actor.

2. Select the Actor, select

Add Component, and choose Stereo Layer.

3. Under the Stereo Layer options, set

Stereo Layer Type to Quad Layer or Cylinder Layer.

4. Set

Stereo Layer Type to Face Locked, Torso Locked, or World Locked.

34 | Advanced Rendering Features | Unreal

5. Set the overlay dimensions in world units in

Quad Stereo Layer Properties or Cylinder Stereo Layer

Properties.

6. Select

Supports Depth in Stereo Layer to set your compositor layer to not always appear on top of other scene geometry. Note that this may affect performance.

7. Configure Texture and additional attributes as appropriate.

The Actor you slave the component to will be fixed at the center of the quad or the cylinder.

You may add up to three VR compositor layers to Gear VR applications, and up to fifteen to Rift applications.

Layers Sample

The LayerSample, available from our Unreal GitHub repository, illustrates the use of VR Compositor Layers to display a UMG UI. For more information, see

Unreal Samples

on page 71.

Multi-View (Mobile)

Multi-view is an advanced rendering feature for Gear VR, available in experimental form in Unreal 4.14 and later. If your application is CPU-bound, we strongly recommend using multi-view to improve performance.

In typical OpenGL stereo rendering, each eye buffer must be rendered in sequence, doubling application and driver overhead. When multi-view is enabled, objects are rendered once to the left eye buffer, then duplicated to the right buffer automatically with appropriate modifications for vertex position and view-dependent variables such as reflection.

Multi-view is currently supported by Note5, S6, S7, and S7 Edge phones using ARM Exynos processors and running Android M or N. It is also supported on S7 and S7 Edge phones using Qualcomm processors and running Android N.

Multi-view requires OpenGL ES 2, and does not currently support OpenGL ES 3.1.

Although multi-view can substantially reduce CPU overhead, keep in mind that applications submitted to the

Oculus Store must maintain minimum frame rate per our requirements, even on devices that do not support multi-view.

Enabling Multi-View

Open

Edit > Project Settings > Engine > Rendering. In the VR section, enable Mobile Multi-View.

For Exynos devices, verify that

Support OpenGL ES2 is checked in the Build section in Platforms > Android, and that

Support OpenGL ES3 is not selected.

Direct Multi-View

Oculus versions 4.15 and higher using integration 1.12 or later support an enhanced version of multi-view that reduces the number of full-screen copies that must be rendered. In most cases, it provides a substantial reduction in GPU overhead.

In Unreal 4.15, to enable Direct Multi-View, add the following to defaultengine.ini:

[Oculus.Settings] bDirectMultiview=True

In Unreal 4.16 and later, when you enable multi-view as described above, you will have a checkbox option to set it to Direct Multi-View.

Unreal | Advanced Rendering Features | 35

Hybrid Monoscopic Rendering (Mobile)

Hybrid monoscopic rendering (available in Unreal 4.15 or later) renders objects close to the viewer in stereoscopic 3D, while rendering all objects that lie past a culling plane only once.

Overview

Enabling hybrid monoscopic rendering can produce performance improvements by drawing many objects only once instead of twice at a slight stereo disparity. In one test scenario, we measured a 25% decrease in rendering times on Epic’s SunTemple sample. In most cases, the visual effect of rendering objects monoscopically with default settings is unnoticeable for typical users.

When hybrid monoscopic rendering feature is enabled, a third monoscopic camera is placed between the left and right cameras on the same plane. In mobile applications, the eye cameras have symmetric frusta, so the monoscopic camera shares the same projection matrix as the stereoscopic cameras.

The UE4 mobile forward renderer then does the following:

1. Renders non-transparent content with the stereo cameras.

2. Shifts and combines the output to create a monoscopic occlusion mask, which pre-populates the monoscopic depth buffer.

3. Renders non-transparent content with the monoscopic camera.

4. Composites the monoscopic camera’s result into the stereo buffers.

5. Render all transparent content and perform all post processing in stereo.

To separate stereo and mono content, hybrid monoscopic rendering uses a depth buffer. All pixels beyond the culling plane are discarded in the stereo view by clearing the stereo depth buffer to that depth. The monoscopic projection near plane is also initialized at that depth to discard fragments that have already been rendered in stereo.

For additional technical details, see

Hybrid Mono Rendering in UE4 and Unity

in our Developer Blog.

Enabling Hybrid Monoscopic Rendering

Open

Edit > Project Settings > Engine > Rendering. In the VR section, enable Monoscopic Far Field

(Experimental).

36 | Advanced Rendering Features | Unreal

Culling Plane

When hybrid monoscopic rendering is enabled, a split plane parallel to the z-axis is added to the level, and objects falling on the far side of the plane are rendered using the monoscopic camera. Objects straddling the culling plane are rendered by both the monoscopic and stereoscopic cameras - see

Best Practices below for more information.

The distance of the culling plane may be configured in

Settings > World Settings > VR > Mono Culling

Distance (default setting is 750.0, or 7.5 meters).

Console Commands

When hybrid monoscopic rendering is enabled, you may set its mode with the following console command: vr.MonoscopicFarFieldMode [0-4]

Mode Description

Off

On (default)

Stereo only

Stereo only with no culling plane

Mono only

Best Practices

This section describes how to implement and tune hybrid monoscopic rendering. We recommend reading it carefully - the substantial performance improvements this feature provides when properly implemented make it

Unreal | Advanced Rendering Features | 37

worth spending some time to get it working properly. However, if it is improperly used, it may actually decrease performance.

Due to limitations of the frustum culling algorithm we use to cull meshes, large objects around the scene such as environment cubemaps or skyboxes are still drawn to the stereoscopic cameras that have a close culling plane, even if no pixel of those objects makes it past the far plane. Those draw calls are unnecessary and create bandwidth and vertex costs to the GPU and CPU.

To prevent unnecessary draw calls in the stereo buffer, we strongly recommend that you identify objects that are further than the culling plane and still render, and force them to render in the monoscopic buffer only.

To identify all objects rendered to the stereoscopic cameras, set your level to display objects drawn to the stereoscopic buffer only with the console command vr.MonoscopicFarFieldMode 2. You will only see objects that lie on the near side of the culling plane, like this:

Figure 1: Stereo Only

38 | Advanced Rendering Features | Unreal

Next, set your level to display all draw calls to the stereoscopic buffer with the console command vr.MonoscopicFarFieldMode 3. This will display the stereo buffer without culling:

Figure 2: Stereo Only with No Culling

Any mesh that appears when vr.MonoscopicFarFieldMode is set to 3 but not when it is set to 2 is redundantly rendered and should be set to render in the monoscopic buffer only. In the pictures above, that includes anything visible in the second image that does not appear in the first image.

Notice in this example the mountains in the scene are drawn to the stereoscopic buffer - this is a typical case.

Any very large object that the user is inside of will typically be a candidate for forcing to mono only.

To force any mesh to render in the monoscopic buffer only, select the mesh and check the

Render in Mono option under Rendering.

Unreal | Advanced Rendering Features | 39

Hybrid monoscopic rendering is usually a better fit for levels that contain a lot of distant content, such as outdoor scenes. In a small room where nearly everything lies before or straddles the culling plane, it will probably not improve performance.

Note that you may enable or disable hybrid monoscopic rendering dynamically in an application, enabling it in levels where it provides a performance improvement, and disabling it when it does not.

One way to quickly assess the performance impact is to use the console command stat RHI to configure your application to display the number of triangles being drawn in your level in real time. Then run the level with hybrid monoscopic rendering disabled and compare it to running the same level with the feature enabled, with different values for the culling plane, with different meshes tagged with

Render in Mono, and so forth.

Adaptive Pixel Density

Adaptive Pixel Density allows applications to scale down the application viewport as GPU resources exceed

85% utilization, and to scale up as they become more available.

For CPU-bound applications, this feature has the potential to improve visual quality.

The following charts illustrate pixel density (gold) and frames per second (blue) on a demo application with

Adaptive Pixel Density off and on, respectively.

40 | Advanced Rendering Features | Unreal

Unreal | Advanced Rendering Features | 41

To enable Adaptive Pixel Density, use the console command hmd pdadaptive on or the console variable vr.oculus.PixelDensity.adaptive on. You may specify a minimum and maximum scaling factor (default

1 = normal density). See

on page 48 for more information.

42 | Advanced Rendering Features | Unreal

To enable Adaptive Pixel Density on startup, specify the appropriate settings in Engine/Config/BaseEngine.ini.

For example:

[Oculus.Settings]

PixelDensityMin=0.5

PixelDensityMax=1.0

PixelDensityAdaptive=true

See

Loading Console Variables

in Epic’s

Console Manager: Console Variables in C++

for more information.

If you do not want some Actors within your level (e.g., text displays) to be scaled, they should be drawn using separate VR Compositor Layers which are not scaled by pixel density. For more information, see

VR Compositor

Layers

on page 32.

Note: To minimize the perceived artifacts from changing resolution, there is a two-second minimum delay between every resolution change.

Unreal | Application Lifecycle Handling | 43

Application Lifecycle Handling

Applications may query for status changes such as losing input focus or the presence of system overlays and take appropriate action such as pausing.

Input Focus

With Oculus integration 1.18 or later, applications may query the

Has Input Focus flag in the Oculus Library, which returns true if the game has input focus. In most cases, when input focus status is lost, the appropriate action is to hide the tracked controllers in the game. For an illustration of a typical case, see the VRCharacter

Blueprint in our Touch sample.

System Overlays

With Oculus integration 1.18 or later, applications may query the

Has System Overlay Present flag in the

Oculus Library, which returns true if a system overlay such as a menu is being displayed. In most cases, when a system overlay is present, the appropriate action is pause and mute the application.

44 | Application Lifecycle Handling | Unreal

For an illustration of a simple case, see the Pause Actor in our Touch sample.

Note: The Pause Actor must continue to tick in the scene, or your application will not be able to detect when the system overlay is no longer present and will never unpause.

The application now shares its depth buffer with the compositor alongside with its color buffer. Depth buffer rendering is enabled by default. To disable depth buffer support, add the following to defaultengine.ini:

[Oculus.Settings] bCompositeDepth=false

If you disable depth buffer handling, we recommend hiding any Actors that may occlude the overlay (i.e., everything within 2-3 meters of the player) when Has System Overlay Present returns true.

Unreal | Oculus Platform Features and Online Subsystems | 45

Achievements

Top Players

Friends

Rooms

Matchmaking

Cloudsaves

Identity

Entitlements

VoIP

The Oculus Platform supports several features related to security, community, revenue, and engagement, including matchmaking, in-app purchase, entitlement checking, VoIP, cloudsaves, and more.

Unreal developers may access Oculus Platform features in two ways:

1. A subset of Platform functionality is built into the engine and is available through the Unreal Online

Subsystem interface, and

2. The Platform SDK includes an Unreal plugin providing full access to all Oculus Platform features

Some Platform features support Blueprints, such as Entitlement Checking. For more information and instructions, see the

Platform section of

on page 51

For instructions on using the Oculus Platform Plugin for Unreal, please see our

Platform SDK documentation

and the readme included in the Unreal folder of the Platform SDK archive (available from our

Unreal Downloads page

Unreal Online Subsystems

Several Oculus Platform features may be accessed through Unreal’s Online Subsystems (OSS) interface in

Unreal Engine 4.12 and 4.13. For more information on available UE4 versions and compatibility, see our

Unreal

Engine

on page 5.

For each supported feature, only a subset of functionality is exposed by the Online Subsystems interface.

Developers who wish to use Platform features not available through Online Subsystems should code against the native C headers located in Engine\Source\ThirdParty\Oculus\LibOVRPlatform and include them in their build.

The following features are available through the Online Subsystems.

Feature Online Subsystem Interface

Achievements

Leaderboards

Sessions

Cloudsaves

Identity

Voice

46 | Oculus Platform Features and Online Subsystems | Unreal

Feature

P2P Networking

Online Subsystem Interface

Netdriver

To enable the OSS plugin, select

Edit > Plugin Settings > Built-In > Online Platform > Online Subsystem

Oculus, and check Enabled.

To access Oculus Platform features through Online Subsystems, you must adjust your defaultEngine.ini file to use Oculus:

[OnlineSubsystem]

DefaultPlatformService=Oculus

[OnlineSubsystemOculus] bEnabled=true

OculusAppId=<app_id_here>

For more information on Unreal Subsystems, see Epic’s

Online Subsystems Overview

Entitlement Checking

Entitlement checking may be used to protect apps from unauthorized distribution. Unreal developers using

Blueprints can use the

Verify Entitlements Blueprint under Oculus > Entitlement. For more information and instructions, see the

Platform section of

These commands are available in Unreal versions using Oculus integration 1.14 or earlier. See

on page 51 and

Entitlements: App Authorizations

in our

Platform guide.

Applications should always have entitlement checks enabled for submissions to the Oculus store.

VoIP

Our Online Subsystems implementation of VoIP extends the Unreal class IOnlineVoice. See Unreal’s

IOnlineVoice

reference for more information.

To enable VoIP in DefaultEngine.ini:

[OnlineSubsystem]

DefaultPlatformService=Oculus bHasVoiceEnabled=true

[Voice] bEnabled=true

To connect to someone else:

Online::GetVoiceInterface()->RegisterRemoteTalker(<FUniqueNetIdOculus>);

To disconnect from someone:

Online::GetVoiceInterface()->UnregisterRemoteTalker(<FUniqueNetIdOculus>);

To see if a remote user is talking:

Online::GetVoiceInterface()->IsRemotePlayerTalking(<FUniqueNetIdOculus>);

To unmute yourself:

Online::GetVoiceInterface()->StartNetworkedVoice(0);

Unreal | Oculus Platform Features and Online Subsystems | 47

To mute yourself:

Online::GetVoiceInterface()->StopNetworkedVoice(0);

P2P

Our Online Subsystem implementation of P2P extends the Unreal class UNetDriver. See Unreal's

UNetDriver

reference for more information.

To enable P2P in DefaultEngine.ini:

[/Script/Engine.GameEngine]

+NetDriverDefinitions=(DefName="GameNetDriver",DriverClassName="OnlineSubsystemOculus.OculusNetDriver",DriverClassNameFallback="OnlineSubsystemUtils.IpNetDriver")

[/Script/OnlineSubsystemOculus.OculusNetDriver]

NetConnectionClassName="OnlineSubsystemOculus.OculusNetConnection"

To act as the host of a game, open the map as a listen server.

To act as the client of the game, open a connection to

<oculus_id_of_host>.oculus.

48 | Unreal Console Variables and Commands | Unreal

Unreal Console Variables and Commands

This document reviews useful console variables and commands available for Unreal development.

Unreal Console Variables (1.15 or later)

Oculus integration 1.15 and later replace our previous console command model with console variables. For a complete description of available console variables, see UE4-Oculus.txt in the root folder of your Unreal installation.

Legacy Unreal Console Commands (1.14 or earlier)

Haptics for Rift Controllers

page 5 for more information.

Rift: Press the tab keys while your game is running to bring up the console.

Gear VR: To bring up the console on a mobile device, set it to developer mode (instructions

https:// developer.oculus.com/documentation/mobilesdk/latest/concepts/mobile-troublesh-device-run-app-outside/

), launch the application, and tap the screen with four fingers.

To specify console commands to be loaded on startup, in most cases you should add them to Engine/Config/

ConsoleVariables.ini. See

Loading Console Variables

in Epic’s

Console Manager: Console Variables in C++

for more information.

For more information, see "Useful VR Console Commands" in Unreal's

VR Cheat Sheet

Table 3: Configuration Commands

Command Description stereo on|off stereo e=0.064

stereo w2m=100 stereo ncp=10 fcp=10000 stereo cs=1 ps=1 stereo show stereo reset hmd enable|disable hmd pd [0..3.0] hmd pdadaptive on|off

Enables/Disables stereo mode.

Eye distance (m).

Overrides default world-units-to-meters scale.

Overrides near clipping and/or far clipping planes for stereo rendering (in cm).

Overrides camera and position scale.

Shows current ipd and head model offset.

Resets stereo settings.

Enables/Disables HMD.

Sets pixel density in the center (default is 1.0).

Enables/Disables adaptive pixel density (see

Adaptive Pixel Density

on page 39 for more details).

Unreal | Unreal Console Variables and Commands | 49

Command hmd pdmax [0.5..2.0] hmd pdmin [0.5..2.0] hmd sp [30..300] hmd hqdistortion on|off hmd mirror on|off hmd mirror mode [0..4] hmdpos on|off|toggle hmdpos enforce on|off hmdpos reset {yaw} hmdpos resetrot {yaw} hmdpos resetpos hmdpos show hmdpos floor|eye

Description

Sets maximum adaptive pixel density (ignored if hmd pdadaptive is off).

Sets minimum adaptive pixel density (ignored if hmd pdadaptive is off).

Overrides screenpercentage for stereo mode. (Deprecated, use 'hmd pd xxx' instead).

Enables/Disables high-quality distortion.

Enables/Disables mirroring to the desktop window.

Sets mirror mode: 0=Distorted, 1=Undistorted, 2=SingleEye, 3=SingleEye Letterboxed,

4=SingleEye Cropped

Enables/Disables/Toggles positional tracking.

Enables/Disables head tracking even if not in stereo (for testing purposes).

Resets position and rotation, applies yaw (in degrees) if provided.

Resets rotation only, applies yaw (in degrees) if provided.

Resets position only.

Outputs status of positional tracking to log.

Selects tracking origin.

Table 4: Misc Commands

Command Description hmd stats hmd grid hmd updateongt on|off hmd updateonrt on|off

Shows HMD-related stats.

Toggles lens-centered grid.

Enables/Disables updating HMD pose on game thread. On by default.

Enables/Disables updating HMD pose on render thread, for lower latency. On by default.

hmd cubemap [gearvr] [xoff=N]

[yoff=N] [zoff=N] [yaw=N]

Generates a cube map image of your application. May be used for VR app previews in the Oculus

Store. Cube map PNGs will be saved in the directory GameDir/Saved/Cubemaps.

gearvr: If specified, cube map size will be 6x1024x1024, otherwise it will be 6*2048x2048.

xoff, yoff, zoff: Offset from the current player's location.

yaw: override yaw rotation (degrees).

50 | Unreal Console Variables and Commands | Unreal

Command Description hmd setint PerfHUDMode [0..4] Selects performance HUD mode, set to 0 to disable.

hmd setint

DebugHudStereoMode [0..3]

Selects debug HUD stereo mode, set to 0 to disable.

hmdversion Prints Oculus SDK version used and Oculus Plugin info.

Unreal | Blueprints | 51

Blueprints

Oculus Blueprints are available for developer use.

To access these Blueprints, you must enable the

Online Subsystem Oculus in Edit > Plugin > Online

Platform.

The Blueprint class Oculus Library provides Blueprints for querying and controlling loading screens, HMD pose, user profile, raw sensor data, position scale, and more.

52 | Blueprints | Unreal

Blueprints available for mobile development include:

• Set CPU And GPU Levels - passes floats to Gear VR Function Library

Several Blueprints useful for mobile VR development are available in Epic's Optional Mobile Features Blueprint

Library, such as:

Unreal | Blueprints | 53

• Are Headphones Plugged In

• Get Battery Level

• Get Battery Temperature

• Get Volume State

Input

The Input Blueprint provides a control interface for Touch, Oculus remote, and the Gear VR Controller.

• Gamepad events

• return Oculus Remote button events (bool)

• return Oculus Touch button events (bool)

• Gamepad Values - return Oculus Touch values (floats)

54 | Blueprints | Unreal

• Oculus Library - provides a number of Blueprints for querying and controlling splash screen, HMD pose, loading icon, user profile, raw sensor data, position scale, and more.

Gear VR Controller clickpad events are reported in the Input Blueprint as thumbstick events. A clickpad click is reported as a press followed by a release. You may query the x- and y-axis of the thumbstick to determine the location of clickpad presses. The Gear VR Controller Back button may be queried with Motion Controller Face

Button 1.

The

Is Device Tracked? Blueprint may be used to infer the handedness of a Gear VR Controller. If a connected input device returns true for Right Controller and false for Left Controller, it is a right-handed controller.

Oculus Touch Haptics

The following Blueprints are not Oculus-specific, but may be used to control haptics for the Xbox or Oculus

Touch controllers. For detailed information, see

on page 23

• Play Haptic Effect

• Stop Haptic Effect

• Set Haptics by Value

• Play Haptic Soundwave

Platform

To access these Blueprints, you must enable the

Online Subsystem Oculus in Edit > Plugin > Online

Platform.

• Verify Entitlements - verifies the user is authorized to run the application.

• Get Oculus Identity - returns Oculus ID and Oculus username as strings; provides flow control for On

Success/On Failure

In this simple example of the Verify Entitlement Blueprint, the application will verify that the user has an entitlement when the application launches, and quit if the check fails. In reality, we recommend providing an appropriate error message before shutting down.

Unreal | Blueprints | 55

This example of the Get Oculus Identity Blueprint illustrates how to print the user’s Oculus ID to the log or screen.

56 | Blueprints | Unreal

Mobile (pre-1.16 integration)

Beginning with Oculus Unreal Integration 1.16, our Unreal integration provides Blueprints based on a common

Oculus API for both Rift and mobile development. Mobile-specific Blueprints based on our previous Gear VR plugin were removed and most functionality was merged into the shared API.

Unreal | Unreal Loading Screens | 57

Unreal Loading Screens

We strongly recommend adding a loading screen to your Rift or mobile application. Loading screens are required by the Oculus Store.

A simple loading screen presenting the user with a rotating texture can be easily added to your game. You will need to specify a texture for display, either with a provided Blueprint or by modifying the defaultengine.ini

configuration file. In both methods, the loading screen is drawn to a VR Compositor Layer, and is guaranteed to render consistently at the required minimum frame rate.

We recommend favoring the configuration file approach over the Blueprint, as Blueprints must be initialized before your Loading Screen can be displayed, and there may be a delay after application launch before they are visible.

In the Oculus branches of Unreal 1.14 and earlier, a rotating arrow loading screen was provided by default.

Versions 1.16 and later do not provide a default loading screen.

Create a Loading Screen with a Configuration File

Add an entry similar to the following example to your project’s defaultengine.ini file. The values given in this example work well for a typical icon.

[Oculus.Splash.Settings]

TexturePath=/Game/LoadingIconTexture.LoadingIconTexture

DistanceInMeters=X=6.0 Y=0.0 Z=0.0

SizeInMeters=X=0.25 Y=0.25

RotationDeltaInDegrees=1.0

RotationAxis=X=0.0 Y=0.0 Z=1.0

Create a Loading Screen using a Blueprint

Oculus provides Blueprints to show, hide, and remove loading screens and transition screens. The Add Loading

Icon Blueprint requires specifying the path of a texture to be added, and will then display that texture in the middle of the screen during loading.

For greater control over how the icon will be displayed, use the Add Loading Splash Screen Blueprint:

58 | Unreal Loading Screens | Unreal

For more information on Oculus Blueprints, see

on page 51.

Unreal | Unreal Mixed Reality Capture | 59

Unreal Mixed Reality Capture

This guide describes how to add and configure mixed reality capture support for your Unreal application.

Mixed reality capture is supported for Rift applications only.

Introduction

Mixed reality capture places real-world people and objects in VR. It allows live video footage of a Rift user to be composited with the output from a game to create combined video that shows the player in a virtual scene.

(Courtesy of Medium and artist Dominic Qwek -

https://www.oculus.com/medium/

)

Live video footage may be captured with a stationary or tracked camera. For more information and complete setup instructions, see the

Mixed Reality Capture Setup Guide

Once configured to use mixed reality capture, applications can be launched with the feature enabled by running them with the appropriate parameter - see "Command-line Parameter Reference" below for more information.

Compositing the Scene

Mixed reality capture supports two methods for combining application output and video footage: direct composition and external composition.

In direct composition mode, your mixed reality capture application streams the real-world footage from your camera to your scene directly, and displays the composited image in the application itself. Direct composition

60 | Unreal Mixed Reality Capture | Unreal

mode requires the use of a green screen for video capture, and the composited image may exhibit some latency from the video stream. We currently recommend using it for proof-of-concept, troubleshooting, and hobby use.

For more polished composition, we recommend using external composition mode. In this mode, the application outputs two windows. The MirrorWindow displays the application, as shown below. The second window displays the foreground content from the video stream on the left against a green background, and it displays the background content on the right.

Figure 3: External Composition Mode

In external composition mode, third-party composition software such as OBS Studio or XSplit is required to clip the green screen, combine the images, and compensate for camera latency.

For more information on how to composite a scene, see the

Mixed Reality Capture Setup Guide

Preparation

You must run the CameraTool prior to launching your mixed reality capture application to configure the external camera and VR Object. See the

Mixed Reality Capture Setup Guide

for setup information.

Create Simple VR Application (Optional)

If you wish to experiment with this feature but do not have a VR application prepared, create a basic application with the following steps.

1. Create a new project with the Virtual Reality Blueprint template.

2. Open the MotionControllerMap in

Content > VirtualRealityBP > Maps.

Unreal | Unreal Mixed Reality Capture | 61

3. Configure the Virtual Reality Blueprint for mixed reality capture.

a. (Optional) To be able to use a Touch controller as an input device with the Virtual Reality Blueprint, open the Project Settings menu, select

Engine - Input and expand Action Mappings. Modify the settings as shown below.

b. (Optional)

Open Edit Preference in the General - Miscellaneous panel and uncheck Use Less CPU when in Background. This prevents applications with mixed reality capture enabled from entering a low-

FPS mode when switching to the composition software.

c. The MotionControllerPawn Blueprint contains a bug which sets the Tracking Origin to "Eye Level" incorrectly. Add a link between "Default" and "Set Tracking Origin (Floor Level) as shown:

62 | Unreal Mixed Reality Capture | Unreal

Add Mixed Reality Capture Support

To enable mixed reality capture, you will add a camera actor to your map, and set it as the origin for your tracking space (optional). See the "Blueprint Reference" below for detailed information on the various components and settings.

1. In the Level editor: a. Add an instance of "Oculus MR Casting Camera Actor" to the map, which will be used in mixed reality capture. The composition parameters for this view may be set in the

Oculus MR section of the

OculusMR_CastingCameraActor instance.

2. In the Level Blueprint editor:

Unreal | Unreal Mixed Reality Capture | 63

64 | Unreal Mixed Reality Capture | Unreal

a. Set VRPawn’s VROrigin (or the tracking origin component in your map) as the

OculusMR_CastingCameraActor1’s TrackingReferenceComponent. If you plan to use the first

PlayerController’s position as the tracking reference, you may skip this step. See "Blueprint Reference" below for more information.

b. Bind the OculusMR_CastingCameraActor1 to the first TrackedCamera, which was configured with the

CameraTool. The final Blueprint should looks like this:

3. (Optional) Check the

Casting Auto Start checkbox in the Oculus MR section of

OculusMR_CastingCameraActor1 to configure the engine to automatically open the Casting Window on launch. This option is useful for debugging.

Unreal | Unreal Mixed Reality Capture | 65

66 | Unreal Mixed Reality Capture | Unreal

To check that everything is working properly, launch the map in VR Preview mode and verify that the Casting

Window opens with the mixed reality capture content.

Sample Scene

A trivial sample map with mixed reality capture enabled is available in our GitHub repository (access instructions

here

) in Samples/Oculus/MixedRealitySample.

Launch Commands

Once you package the sample scene, you may launch it with these parameters.

You may set your application to launch with mixed reality capture enabled in Blueprints for debugging purposes only. The Blueprint setting bCastingAutoStart is automatically disabled when you build your package.

// launch in direct composition mode

MixedRealitySample.exe -vr -mxr_open_direct_composition

// launch in direct composition mode with MirrorWindow projection

MixedRealitySample.exe -vr -mxr_open_direct_composition -mxr_project_to_mirror_window

// launch in MultiView mode

MixedRealitySample.exe -vr -mxr_open_multiview

// launch in MultiView with MirrorWindow projection

MixedRealitySample.exe -vr -mxr_open_multiview -mxr_project_to_mirror_window

Blueprint Reference

AOculusMR_CastingCameraActor

Properties

Property Description bCastingAutoStart Starts mixed reality capture casting automatically when the level starts. This option is for debugging only, and will be automatically disabled when the game is launched as a standalone package. Use launch commands to launch applications with mixed reality capture enabled.

Set to true to cast to the MirrorWindow. This can simplify window switching, especially on a single-monitor configuration. By default the scene is casted to a standalone window, which offers the most precision in the composition.

When set to true, the bProjectToMirrorWindow is automatically minimized on startup, as the content is now casting to the MirrorWindow.

CompositionMethod MultiView (default): The casting window includes the background and foreground view for external composition.

DirectComposition: The game scene is composited with the camera frame directly.

ClippingReference Specifies the distance from the camera to the mixed reality capture casting background/foreground boundary. Set to CR_TrackingReference to use the distance to the Tracking Reference (recommended for stationary

Unreal | Unreal Mixed Reality Capture | 67

Property Description experiences). Set to CR_Head to use the distance to the HMD (default, recommended for roomscale experiences).

TrackedCamera Information about the tracked camera which this object is bound to.

(optional) If the application uses a VROrigin component to set the tracking space origin, specify that component here. Otherwise the system will use the location of the first PlayerController as the tracking reference.

If true the casting camera will automatically follow the movement of the tracking reference.

If true the casting viewports will use the same resolution as the camera used in the calibration process.

WidthPerView When bUseTrackedCameraResolution is false, sets the width of each casting viewport (foreground, background, or direct composited).

HeightPerView

CapturingCamera

When bUseTrackedCameraResolution is false, sets the height of each casting viewport (foreground, background, or direct composited).

When CompositionMethod is set to DirectComposition, indicates which physical camera device provides the video frame for compositing

CastingLatency When CompositionMethod is set to MultiView, sets the latency of the casting output. This setting may be used to help sync with the camera latency in the external composition application.

ChromaTorelanceA [Green-screen removal] When CompositionMethod is set to

DirectComposition, sets how heavily to weight non-green values in a pixel.

For example, if the character image looks too transparent, you may increase this value to make it more opaque.

ChromaTorelanceB [Green-screen removal] When CompositionMethod is set to

DirectComposition, sets how heavily to weight the green value. If mid-range greens don’t appear to be cut out, increasing B or decreasing A may help.

ChromaShadows [Green-screen removal] When CompositionMethod is set to

DirectComposition, the shadow threshold helps mitigate shadow casting issues by eliminating very dark pixels.

ChromaAlphaCutoff [Green-screen removal] When CompositionMethod is DirectComposition, alpha cutoff is evaluated after chroma-key evaluation and before the bleed test to fully discard pixels with a low alpha value.

Methods

68 | Unreal Mixed Reality Capture | Unreal

Method Description

If there is no calibrated external camera, the TrackedCamera parameters must be set up to match CastingCameraActor placement. It provides an easy way to directly place a stationary casting camera in the level.

When bFollowTrackingReference is false, manually call this method to move the casting camera to follow the tracking reference (i.e., the player).

OpenCastingWindow Opens the casting window.

CloseCastingWindow Closes the casting window.

ToggleCastingWindowToggles the casting window.

FTrackedCamera

Properties

Property

Index

Name

FieldOfView

SizeX, SizeY

Description

>=0: the index of the external camera.

-1: Do not bind to any external camera (i.e., set up to match the manual

CastingCameraActor placement).

The external camera name set through the CameraTool.

Horizontal FOV, in degrees.

Resolution of the camera frame.

• None: stationary camera

• HMD, LTouch, RTouch: HMD or left/right Touch

• DeviceObjectZero: The VR object

CalibratedRotation,

CalibratedOffset

The relative pose of the camera to the attached tracking device.

Equals the absolute pose in the tracking space if

AttachedTrackedDevice==None.

UserRotation.

UserOffset

(optional) Provide user pose to fine tune the relative camera pose at the runtime.

Unreal | Unreal Mixed Reality Capture | 69

Property Description

Use to match the manual CastingCameraActor placement in the level when

Index == -1.

UOculusMRFunctionLibrary

Methods

Method Description

GetAllTrackedCameraRetrieve an array of all tracked cameras which were calibrated through the

CameraTool.

Command-line Parameter Reference

Parameter Description

-mxr_open Automatically open the casting window in the preset composition mode

mxr_open_multiview

Automatically open the casting window in MultiView composition mode.

Automatically open the casting window in DirectCompositon mode.

mxr_open_direct_composition

Project the casting output to the MirrorWindow.

mxr_project_to_mirror_window

Console commands

Command Description mr.AutoOpenCasting

[0|1|2]

Automatically open the casting window: 0=Off; 1=MultiView;

2=DirectComposition mr.ChromaTorelanceA

<float>

[Green-screen removal] When CompositionMethod is set to

DirectComposition, sets how heavily to weight non-green values in a pixel.

For example, if the character image looks too transparent, you may increase this value to make it more opaque.

mr.ChromaTorelanceB

<float>

[Green-screen removal] When CompositionMethod is set to

DirectComposition, sets how heavily to weight the green value. If mid-range greens don’t appear to be cut out, increasing B or decreasing A may help.

mr.ChromaShadows

<float>

[Green-screen removal] When CompositionMethod is set to

DirectComposition, the shadow threshold helps mitigate shadow casting issues by eliminating very dark pixels.

70 | Unreal Mixed Reality Capture | Unreal

Command Description mr.ChromaAlphaCutoff

<float>

[Green-screen removal] When CompositionMethod is DirectComposition, alpha cutoff is evaluated after chroma-key evaluation and before the bleed test to fully discard pixels with a low alpha value.

mr.CastingLantency

<float>

The casting latency in MultiView mode.

mr.ProjectToMirrorWindow

[0|1]

Project the casting output to the MirrorWindow.

Unreal | Unreal Samples | 71

Unreal Samples

Oculus provides samples which illustrate basic VR concepts in Unreal such as Touch, haptics, and the Boundary

Component API for interacting with the Guardian System.

Samples are available from the

Oculus Unreal GitHub repository

. To access this repository, you must hold a UE4 license and be subscribed to the private EpicGames/UnrealEngine repository (see

https:// www.unrealengine.com/ue4-on-github

for details).

All samples require a compatible version of the Unreal Engine which supports the illustrated features. To explore samples, we generally recommend using Unreal versions that we ship from our GitHub repository, which always include the latest features. For a review of which Unreal versions support which features, see

LayerSample is a Blueprint sample that illustrates the use of

on page 5.

Boundary Sample

BoundarySample is a Blueprint sample that illustrates the use of the

Boundary Component API

for interacting with our Guardian System. This API allows developers to query the Guardian System trigger state, return the closest boundary point to a queried device or point, and more.

In this sample, the Guardian System Boundary Area is visualized on the ground in orange-yellow, and the Play

Area is visualized on the ground in purple.

Two spheres track the Touch controllers. When they approach the Boundary Area, they project a colored arrow toward the nearest sample point in the Guardian Area.

72 | Unreal Samples | Unreal

Unreal | Unreal Samples | 73

NewGameMode and VRCharacter are used to initialize the scene and make the scene display at the appropriate height, et cetera.

Controls and Behavior

These behaviors are controlled by the BoundarySampleMap, which you can use as a reference for implementing related functionality.

Test Control or Behavior

Press 'V' on keyboard/right Touch button A to request boundary on. Press 'X'/right Touch button B to cancel.

Press 'G' on keyboard/left Touch button X to set boundary to green. Press 'P'/left Touch button Y to make purple.

If so, cone is not visible (cone should remain if boundaries are requested on by application).

OnOuterBoundaryTriggered

Event

Cube made visible.

OnOuterBoundaryReturned

Event

Cube made invisible.

Layer Sample

VR Compositor Layers

to display a

UMG UI

This sample includes two spheres that track with the Touch controllers and two UMG widgets rendered as VR

Compositor layers. One is rendered as a quad layer and the other as a cylinder layer.

74 | Unreal Samples | Unreal

Actor_Blueprint illustrates rendering a UMG widget into a stereo layer. The widget is first rendered into a

Material, then the SlateUI texture is pulled from the Material into the stereo layer. This is the UMG widget that is rendered to the quad and cylindrical layers in the sample.

Open MenuBlueprint to open the UMG widget in the

UMG Editor

NewGameMode and VRCharacter are used to initialize the scene and make the scene display at the appropriate height, et cetera.

Unreal | Unreal Samples | 75

Input Sample

InputSample illustrates basic use of Oculus controllers including Touch and Gear VR Controller. It illustrates tracking, thumbstick control, and

haptics

control using PlayHapticEffect() and PlayHapticSoundWave().

Two spheres track with the Touch controllers. The right controller thumbstick may be used to control the position of the light gray box. Touch capacitance sensors detect whether the right thumb is in a touch or neartouch position, which controls the height of the box. Press and hold the left Touch grip button to play a haptics clip. Press and hold the left Touch X button to create a haptics effect by setting the haptics value directly.

76 | Unreal Samples | Unreal

You will find the Haptics control Blueprint and the Thumbstick control Blueprint in the sample Level Blueprint.

NewGameMode and VRCharacter are used to initialize the scene and make the scene display at the appropriate height, et cetera.

Unreal | Unreal Samples | 77

Mixed Reality Capture Sample

A trivial sample map with mixed reality capture enabled is available in our GitHub repository (access instructions

here

) in Samples/Oculus/MixedRealitySample. Select the OculusMR_CastingCameraActor1 instance to see how it is configured for the Level.

For more information, see

Unreal Mixed Reality Capture

on page 59.

Sample Scene

A trivial sample map with mixed reality capture enabled is available in our GitHub repository (access instructions

here

) in Samples/Oculus/MixedRealitySample.

Legacy Touch Sample

This is sample is available with legacy versions using Oculus integration 1.14 or earlier - for more information,

see

control using PlayHapticEffect() and PlayHapticSoundWave(). Two spheres track with the

on page 5.

78 | Unreal Samples | Unreal

TouchSample illustrates basic use of Oculus Touch including controller tracking and thumbstick control. It also

illustrates

haptics

Touch controllers. The right controller thumbstick may be used to control the position of the light gray box.

Touch capacitance sensors detect whether the right thumb is in a touch or near-touch position, which controls the height of the box. Press and hold the left Touch grip button to play a haptics clip. Press and hold the left

Touch X button to create a haptics effect by setting the haptics value directly.

You will find the Haptics control Blueprint and the Thumbstick control Blueprint in the sample Level Blueprint.

NewGameMode and VRCharacter are used to initialize the scene and make the scene display at the appropriate height, et cetera.

Unreal | Testing and Performance Analysis in Unreal | 79

Testing and Performance Analysis in Unreal

This guide describes basic testing and perfromance analysis for Oculus development in Unreal.

VR application debugging is a matter of getting insight into how the application is structured and executed, gathering data to evaluate actual performance, evaluating it against expectation, then methodically isolating and eliminating problems.

When analyzing or debugging, it is crucial to proceed in a controlled way so that you know specifically what change results in a different outcome. Focus on bottlenecks first. Only compare apples to apples, and change one thing at a time (e.g., resolution, hardware, quality, configuration).

Always be sure to profile, as systems are full of surprises. We recommend starting with simple code, and optimizing as you go - don’t try to optimize too early.

Performance Targets

Before debugging performance problems, establish clear targets to use as a baseline for calibrating your performance.

These targets can give you a sense of where to aim, and what to look at if you’re not making frame rate or are having performance problems.

Below you will find some general guidelines for establishing your baselines, given as approximate ranges unless otherwise noted.

Mobile

• 60 FPS (required by Oculus Store)

• 50-100 draw calls per frame

• 50,000-100,000 triangles or vertices per frame

• 90 FPS (required by Oculus Store)

• 500-1,000 draw calls per frame

• 1-2 million triangles or vertices per frame

For more information, see:

•

PC SDK Developer Guide

•

Mobile VR Application Development

• Oculus Store Submission Requirements

•

Oculus Rift App Requirements

•

Gear VR App Requirements

Rift Performance HUD

The Oculus Performance Heads-Up Display (HUD) is an important, easy-to-use tool for viewing timings for render, latency, and performance headroom in real-time as you run an application in the Oculus Rift. The

HUD is easily accessible through the Oculus Debug Tool provided with the PC SDK. You may activate it in the

Viewport by pressing the ~ key.

For more details, see the

Performance Heads-Up Display

and

Oculus Debug Tool

sections of the Oculus Rift

Developers Guide.

80 | Testing and Performance Analysis in Unreal | Unreal

Rift Compositor Mirror

The compositor mirror is an experimental tool for viewing exactly what appears in the headset, with

Asynchronous TimeWarp and distortion applied.

The compositor mirror is useful for development and troubleshooting without having to wear the headset.

Everything that appears in the headset will appear, including Oculus Home, Guardian boundaries, ingame notifications, and transition fades. The compositor mirror is compatible with any game or experience, regardless of whether it was developed using the native PC SDK or a game engine.

For more details, see the

Compositor Mirror

section of the PC SDK Guide.

Debug Console (Mobile)

If your phone is set to Developer Mode, you may bring up a debug console for VR apps by pressing the screen with four fingers simultaneously while the application is running.

Enter stat unit in the console for information about your application frame rate and CPU performance.

Oculus Remote Monitor (Mobile)

The Oculus Remote Monitor client connects to VR applications running on remote mobile devices to capture, store, and display the streamed-in data. The VrCapture library is automatically included in Unreal projects, so setup and use of the Oculus Remote Monitor is easy.

Unreal | Testing and Performance Analysis in Unreal | 81

Oculus Remote Monitor is available from our

Downloads

page. For more information about setup, features, and use, see

Oculus Remote Monitor

in our Mobile SDK guide.

82 | Testing and Performance Analysis in Unreal | Unreal

• The Frame Buffer Viewer provides a mechanism for inspecting the frame buffer as the data is received in real-time, which is particularly useful for monitoring play test sessions. When enabled, the Capture library will stream a downscaled pre-distortion eye buffer across the network.

• The Performance Data Viewer provides real-time and offline inspection of the following on a single, contiguous timeline:

• CPU/GPU events

• Sensor readings

• Console messages, warnings, and errors

• Frame buffer captures

• The Logging Viewer provides raw access to various messages and errors tracked by thread IDs.

• Nearly any constant in your code may be turned into a knob that can be updated in real-time during a play test.

Graphics Debugging

Mali Graphics Debugger

If you have a Mali phone, such as a GALAXY S6, you can use the Mali Graphics Debugger built into Unreal by selecting it by opening

Project Settings, selecting the Android option on the left, and setting Graphics

Debugger to Mali Graphics Debugger.

Note that because there are no swap buffers in VR, Gear VR does not currently support frame delimiters.

Consequently, application frames will be displayed as different render passes of the same frame.

RenderDoc

Version 1.16 and later of the Oculus branch of Unreal provides RenderDoc support.

Gear VR support in RenderDoc is an experimental feature, so you must download a nightly build of RenderDoc to access it. To verify that you have the right version, confirm that your RenderDoc installation includes an android/ subfolder in its root path.

Unreal | Testing and Performance Analysis in Unreal | 83

To use RenderDoc, open

Project Settings in Unreal and select the Android option on the left. Set Graphics

Debugger to RenderDoc.

The RenderDocCmd.apk application must be installed on the target device before debugging. Run the following command from the android/apk/32 directory of your RenderDoc installation: adb install -r RenderDocCmd.apk

If your phone is not auto-recognized as a Remote server in the Tools > Manage Remote Server section, add it by typing adb:<device-id> in the Add section.

To attach to an application, run it and locate it in

Tools > Manage Remote. To capture a frame, click on the

Trigger Capture button. Save the capture locally and double-click it in RenderDoc to replay.

GDB Debugging (Mobile)

Oculus branches of Unreal add support for debugging mobile sessions using ndk-gdb, a small shell script wrapped around GNU GDB that is included with the Android NDK.

Using ndk-gdb from the command line adds convenient features to your debugging workflow by allowing, for example, adding breakpoints, stepping through code, and inspecting variables with a command line interface.

To use ndk-gdb for debugging:

1. Enable remote port forwarding to link your target mobile port to a PC port: adb forward tcp:$port tcp:$port

2. Set your device to Developer Mode (as described in our Mobile Developer Guide

here

3. Launch the application you wish to debug.

4. Start gdbserver on the mobile device with the following Unreal console command: gdbserver $port where

$port is your port number. You application should freeze, and will now be ready for debugging.

5. Launch the ndk-gdb client from the command line on your computer with the command gdb.exe. When it launches, type target remote :$port in the GDB command line to attach to your mobile device.

For more information on using GDB for debugging, see the

GNU GDB documentation

Additional Third-Party Tools

ETW + GPUView

Event Tracing for Windows

(ETW) is a trace utility provided by Windows for performance analysis.

GPUView

view provides a window into both GPU and CPU performance with DirectX applications. It is precise, has low overhead, and covers the whole Windows system. Custom event manifests.

ETW profiles the whole system, not just the GPU. For a sample debug workflow using ETW to investigate queuing and system-level contention, see Example Workflow: PC below.

Windows 10 replaces ETW with

Tracelogging

Systrace

Reports complete Android system utilization. Available here:

http://developer.android.com/tools/help/ systrace.html

NVIDIA NSight

NSight is a CPU/GPU debug tool for NVIDIA users, available in a

Visual Studio version

and an

Eclipse version

Mac OpenGL Monitor

84 | Testing and Performance Analysis in Unreal | Unreal

An OpenGL debugging and optimizing tool for OS X. Available here:

https://developer.apple.com/library/mac/ technotes/tn2178/_index.html#//apple_ref/doc/uid/DTS40007990

APITrace

https://apitrace.github.io/

Other Resources

For detailed information about Oculus development, go to:

• Unreal: Virtual Reality Development:

https://docs.unrealengine.com/latest/INT/Platforms/VRZ/

• Unreal: Oculus Rift wiki:

https://wiki.unrealengine.com/Oculus_Rift

• Oculus Forums/Unreal:

https://forums.oculus.com/developer/categories/unreal

• Unreal Forums/VR Development:

https://forums.unrealengine.com/forumdisplay.php?27-VR-Development

Contact

Visit our developer support forums at

https://developer.oculus.com

Our Support Center can be accessed at

https://support.oculus.com

Unreal | Release Notes | 85

Release Notes

This section describes changes for each version release.

Oculus Unreal Engine 4 Integration 1.18 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.17 available from the Oculus GitHub repository.

1.18.0

New Features

• Added Input Focus and System Overlay support - see

Application Lifecycle Handling

on page 43 for more information.

• Added

Has Input Focus flag to Oculus Library. When an application loses input focus, applications should typically hide tracked controllers in the scene.

• Added

Has System Overlay Present flag to Oculus Library. When system overlay status changes, applications should typically pause and mute.

• Added depth buffer sharing with the compositor. Depth buffer rendering is enabled by default.

• Touch Sample

• Added VRCharacter Blueprint illustrating how to hide tracked controllers when an application loses input focus.

• Added Pose Actor illustrating how to pause a simple application using SystemOverlayPresent flag.

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus UE4 1.15 and earlier: Oculus Blueprints not visible in Level Blueprint in Epic launcher and

Epic source versions when Gear VR Plugin is enabled. Workaround for source version: open the file

GearVR.uplugin and replace "WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android",

"Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

• UE4 Issue - UE4 builds fail when using Android SDK tools 25 or newer. Please review the

UE Answers

page related to this issue for more information.

• Hybrid Monoscopic Rendering: In Unreal version 4.15, this feature is available for mobile only, and may not be used with Multi-view.

86 | Release Notes | Unreal

Oculus Unreal Engine 4 Integration 1.17 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.17 available from the Oculus GitHub repository.

1.17.0

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus UE4 1.15 and earlier: Oculus Blueprints not visible in Level Blueprint in Epic launcher and

Epic source versions when Gear VR Plugin is enabled. Workaround for source version: open the file

GearVR.uplugin and replace "WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android",

"Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

• UE4 Issue - UE4 builds fail when using Android SDK tools 25 or newer. Please review the

UE Answers

page related to this issue for more information.

• Hybrid Monoscopic Rendering: In Unreal version 4.15, this feature is available for mobile only, and may not be used with Multi-view.

Oculus Unreal Engine 4 Integration 1.16 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.16 available from the Oculus GitHub repository.

1.16.0

Oculus-1.16 adds support for mixed reality capture (Rift only), which places real-world objects in VR. It allows live video footage of a Rift user to be composited with the output from a game to create combined video that showed the player in a virtual scene. For more information, see

Unreal Mixed Reality Capture

on page 59.

New Features

• Oculus-1.16 adds support for mixed reality capture (Rift only), which allows live video footage of a Rift user to be composited with the output from a game to create combined video that showed the player in a virtual

scene. For more information, see

Unreal Mixed Reality Capture

on page 59.

• Added sample map with mixed reality capture enabled to Collaboration\Oculus\Public\MixedRealitySample.

• Refactored Blueprints for unified Oculus plugin model. See

on page 51 for more information.

• Added Blueprints for simplified Loading Screen support.

• Added Android graphics debugging support to our 4.16 branch. See

Testing and Performance Analysis in

Unreal

on page 79 for more information.

• 4.16-oculus added RenderDoc graphics debugging support (Android only, experimental)

Unreal | Release Notes | 87

• 4.16-oculus added Mali Graphics Debugger support (Android only, requires a Mali phone such as the

Android GALAXY S6).

API Changes

• Removed default loading screen. Use defaultengine.ini or loading screen Blueprints to add a loading screen to your application. See

Unreal Loading Screens

on page 57 for more information.

• Added anyhand setting to motion controller component. It will attach to any Gear VR Controller, whether left-handed or right-handed.

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus UE4 1.15 and earlier: Oculus Blueprints not visible in Level Blueprint in Epic launcher and

Epic source versions when Gear VR Plugin is enabled. Workaround for source version: open the file

GearVR.uplugin and replace "WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android",

"Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

• UE4 Issue - UE4 builds fail when using Android SDK tools 25 or newer. Please review the

UE Answers

page related to this issue for more information.

Oculus Unreal Engine 4 Integration 1.15 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.15 and 4.16 available from the Oculus

GitHub repository.

1.15.0

OculusVR Plugin

With the 1.15 integration we are releasing versions of 4.15 and 4.16 that include a substantial refactor of our

Unreal plugins. Our OculusRiftHMD, GearVR, OculusInput, and OculusFunctionLibrary plugins have been combined into a unified OculusVR plugin. All functionality may be accessed by enabling the OculusVR plugin.

This plugin refactor will provide a consistent and unified interface to develop against for all current and future

Oculus devices. All feature development will be based on the OculusVR plugin going forward.

Note that in the 4.16 version no longer supports the exec-style “hmd …” console commands - they have been replaced by console variables. See “UE4-Oculus.txt” in the root directory for a description of available console variables.

Please let us know if you have any questions, suggestions, or problems in our

Unreal Developer Forum

New Features

• Added Rift support to cylinder VR Compositor Layers.

Known Issues

88 | Release Notes | Unreal

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus Blueprints not visible in Level Blueprint in Epic launcher and Epic source versions when Gear

VR Plugin is enabled. Workaround for source version: open the file GearVR.uplugin and replace

"WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android", "Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

• UE4 Issue - UE4 builds fail when using Android SDK tools 25 or newer. Please review the

UE Answers

page related to this issue for more information.

Oculus Unreal Engine 4 Integration 1.14 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.14, 4.15, and 4.16 available from the Oculus

GitHub repository.

1.14.0

The Oculus Avatar SDK now provides a plugin for Unreal to assist developers with implementing first-person hand presence for the Rift and Touch controllers. It includes avatar hand and body assets viewable by other users in social applications for Rift and Gear VR. For installation and use instructions, please see

Getting Started with Unreal

in our Avatar documentation.

Preview of Unified OculusVR Plugin

With the 1.14 integration we are releasing versions of 4.15 and 4.16 that include a preview of a substantial refactor of our Unreal plugins. Our OculusRiftHMD, GearVR, OculusInput, and OculusFunctionLibrary plugins have been combined into a unified OculusVR plugin. All functionality may be accessed by enabling the

OculusVR plugin.

This plugin refactor will provide a consistent and unified interface to develop against for all current and future

Oculus devices. All feature development will be based on the OculusVR plugin going forward.

Preview releases containing the unified OculusVR plugin are available here:

•

Unreal 4.15 Unified

•

Unreal 4.16 Unified

Access requires a GitHub account subscribed to the private EpicGames/UnrealEngine repository (instructions

here

). If you are not logged into a subscribed account, you will get a 404 error from GitHub.

Please let us know if you have any questions, suggestions, or problems in our

Unreal Developer Forum

New Features

• Online Subsystems

Unreal | Release Notes | 89

• Added a warning if the message task that handles popping OVR messages does not run in four seconds

(it should run every tick).

• If the initial NetDriver connection times out, it now retries until the amount of time specified by

InitialConnectTimeout in DefaultEngine.ini passes.

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus Blueprints not visible in Level Blueprint in Epic launcher and Epic source versions when Gear

VR Plugin is enabled. Workaround for source version: open the file GearVR.uplugin and replace

"WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android", "Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

Oculus Unreal Engine 4 Integration 1.13 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.14 and 4.15 available from the Oculus

GitHub repository.

1.13.0

New Features

• 4.15: Added GearVRControllerComponent, which creates a MotionController with a Gear VR Controller mesh as a child. The Gear VR Controller mesh may be found in Plugins/GearVR/Content/.

Bug Fixes

• Fixed black screen issue caused by lowering pixel density to certain values on Qualcomm devices.

• Fixed a race condition causing the GearVRController plugin to crash while the game was shutting down.

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus Blueprints not visible in Level Blueprint in Epic launcher and Epic source versions when Gear

VR Plugin is enabled. Workaround for source version: open the file GearVR.uplugin and replace

"WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android", "Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

90 | Release Notes | Unreal

Oculus Unreal Engine 4 Integration 1.12 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.14 and 4.15 available from the Oculus

GitHub repository.

1.12.0

New Features

• Added support for Gear VR Controller. See

Gear VR Controller

on page 23 for more information.

• Added Direct Multi-View support, which can substantially reduce GPU overhead. See

Multi-View (Mobile)

on page 34 for more information.

Bug Fixes

• Fixed Adaptive Pixel Density flickering when ASW is enabled.

Known Issues

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Setting the mirror window to full-screen exclusive mode will not work correctly if the monitor and HMD are connected to different GPUs.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus Blueprints not visible in Level Blueprint in Epic launcher and Epic source versions when Gear

VR Plugin is enabled. Workaround for source version: open the file GearVR.uplugin and replace

"WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android", "Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

Oculus Unreal Engine 4 Integration 1.11 Release

Notes

These release notes describe changes to Oculus's Unreal Engine 4.13, 4.14, and 4.15 available from the Oculus

GitHub repository.

1.11.0

Oculus distributes integrations of the Unreal Engine which include the latest Oculus SDKs to provide up-to-date support for Oculus Rift and Gear VR.

New Features

• Added GDB debugging support. See “GDB” in

Testing and Performance Analysis in Unreal

on page 79 in

Unreal for more information.

• Added three Blueprint samples, available from our

GitHub repository

. For more on how to access our repository, see our Developer Guide Introduction.

• BoundarySample illustrates use of the

Boundary Component API

for interacting with our Guardian

System.

• LayerSample illustrates the use of

VR Compositor Layers

Unreal | Release Notes | 91

• TouchSample illustrates basic use of Oculus Touch, including

haptics

• Online Subsystems

• LeaderboardInterface now allows for querying of just the user's entry by calling ReadLeaderboards() with an array of just the current user id.

• SessionInterface

• Added the ability to populate the LocalOwnerId with the current user.

• DumpSessionState() now dumps out all individual existing named sessions.

Bug Fixes

• Added support for wire frame representation of objects while using the 11:11:10 LDR format

• Online Subsystems

• AchievementInterface: Fixed casting WriteAchievement for Count values for Int64, UInt32, and UInt64.

• Fixed bug preventing the UE4 Read Leaderboard blueprint node from working with the Oculus OSS.

• SessionInterface: Fixed a NullPointerException when getting the oculus id from a session

• Allow for updating the Room Data Store through the UpdateSession(). Will save the UE4 Session

Settings into the Room Data Store.

• Fixed a bug causing TriggerMatchmakingCompleteDelegates to fire too early

• Session.bHosting now returns if the current user is the host/owner of the session.

Known Issues

• Adaptive Pixel Density is not currently working and should be disabled. If applications using Adaptive Pixel

Density reach 45 Hz, they will remain stuck at that frame rate until relaunched. A fix is planned for the Rift

1.12 runtime release and should not require any application changes.

• A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Exclusive Mode issues: Multiple initializations of the DXGISwapChain may cause flickering as the screen switches modes and a black screen when rendering to the Rift with a different GPU from the one the game is using to render the eye buffers.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

• Oculus Blueprints not visible in Level Blueprint in Epic launcher and Epic source versions when Gear

VR Plugin is enabled. Workaround for source version: open the file GearVR.uplugin and replace

"WhitelistPlatforms" : ["Android"] with "WhitelistPlatforms" : [ "Android", "Win64", "Win32" ]. This does not affect source shipped through the Oculus Unreal GitHub repository.

Oculus Unreal Engine 4 Integration 1.10 Release

Notes

1.10.0

These release notes describe changes to Oculus's Unreal Engine 4.12, 4.13, and 4.14 available from the Oculus

GitHub repository.

New Features

92 | Release Notes | Unreal

• Added experimental multi-view support for mobile development, which reduces CPU overhead by duplicating objects to both eye buffers during rendering. Currently supports Note5, S6, S7, and S7 Edge phones running Android M or N and using ARM Exynos processors. Requires OpenGL ES 2. For more information, see

Multi-View in

Unreal Mobile Development

on page 18.

• Added “Browse” matchmaking support in OnlineSessionOculus.

• Added ability to create a matchmaking room.

• Added ability to find matchmaking rooms in Browse mode.

• Added Blueprints to Create and Find matchmaking rooms.

Bug Fixes

• Fixed Oculus Blueprints error calling

Activate() twice.

• Fixed crash when Oculus Cloud Saves used when the application is not installed.

Known Issues

• A significant drop in framerate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit > Editor Preferences > General (left sidebar) >

Miscellaneous (left sidebar) > Performance.

• Stereo Layer Depth Ordering: Doesn’t support head-locked layers, only world-locked and tracker-locked.

Oculus Unreal Engine 4 Integration 1.9 Release

Notes

1.9.0

These release notes describe changes to Oculus's Unreal Engine 4.12 and 4.13 available from the Oculus

GitHub repository.

New Features

• Added VoIP and moderated room finding/joining support, available through the Online Subsystems

interface. For more information, see

Oculus Platform Features and Online Subsystems

on page 45.

Bug Fixes

• Fixed near clipping plane calculation, which was incorrect when WorldToMetersScale was set to a value other than 100.

Known Issues

• UE4.12 and 4.13 in Oculus GitHub repository: A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit >

Editor Preferences > General (left sidebar) > Miscellaneous (left sidebar) > Performance.

•

Stereo Layer Position Type: World Locked is not currently supported for cylinder TimeWarp overlays.

Unreal | Release Notes | 93

Oculus Unreal Engine 4 Integration 1.8 Release

Notes

1.8.0

These release notes describe changes to the Unreal Engine versions 4.12 and 4.13 available from the Oculus

GitHub repository.

The Oculus UE4 integration ships with various versions of Unreal, and there is no external SDK per se. Clicking on the "Download" link above will redirect you to our

on page 5, which walks through various

options for working with the Oculus Unreal integration.

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows, and Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

New Features

• Added support for the Oculus Guardian System, which visualizes the bounds of a user-defined Play Area.

Note that it is currently unsupported by public versions of the Oculus runtime. See

Guardian System

Boundary Component

on page 28 for more information.

• Added depth ordering support to VR compositor layers.

• Added cubemap support for VR compositor overlays to 4.13 (mobile only).

• Added Online Subsystem support for Regular Leaderboards and Cloudsaves.

• Added a Blueprint for retrieving Oculus ID/Username.

• Added Blueprints for

UOculusRiftBoundaryComponent public methods.

API Changes

• Added OculusRiftBoundaryComponent API for the Oculus Guardian System.

Bug Fixes

• Fixed black screen issue affecting Samsung Note4 and certain Adreno-based devices.

Known Issues

• UE4.12 and 4.13 in Oculus GitHub repository: A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit >

Editor Preferences > General (left sidebar) > Miscellaneous (left sidebar) > Performance.

•

Stereo Layer Position Type: World Locked is not currently supported for cylinder TimeWarp overlays.

94 | Release Notes | Unreal

Oculus Unreal Engine 4 Integration 1.7 Release

Notes

1.7.0

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows.

Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

For more information on downloading and using the Oculus Unreal integration, please see

page 5.

Online Subsystems

Several features of the Oculus Platform SDK may be accessed through the Unreal Online Subsystems interface:

• Achievements

• Rooms

• Matchmaking

• Identity

• Entitlements

• P2P networking

• Friends leaderboards

Friends leaderboards is not currently available through Online Subsystems in Epic’s 4.13 Preview build.

Not all functionality of these features is available through the Online Subsystems interface. Developers who need unsupported features should use the Oculus Platform Plugin available with our Platform SDK.

Online Subsystems access is available in the following ways:

• Oculus Platform SDK v. 1.6 and 1.7 include a standalone OSS plugin (Rift only)

• Epic GitHub: 4.13.0-preview-2 includes OSS support (Rift and Gear VR)

• Oculus GitHub: oculus-4.12.5-release-1.7.0 and oculus-4.13.0-preview-2-1.7.0 includes OSS support (Rift and Gear VR)

For more information on Unreal Subsystems, see Epic’s Online Subsystems Overview.

New Features

• Added adaptive viewport scaling, which automatically changes pixel density based on resource availability, with configurable min/max. See

on page 48 for details. (Rift only)

• Added Oculus Platform SDK support to mobile VR applications to 4.12 and 4.13 in the Oculus repository.

• Added Blueprint for Oculus Platform entitlement checks (available in SI 1.6).

• A subset of Oculus Platform features are now available through the Unreal Online Subsystems interface.

• Added cylinder TimeWarp overlay support (Gear VR only) for UI components, etc. For more information, see

VR Compositor Layers

on page 32.

• Added ability develop using GearVR plugin and Mobile renderer on the PC using the Oculus Rift. To use, disable OculusRift and SteamVR plugins for your project, and add

-OpenGL -FeatureLevelES2 to your command line.

API Changes

Unreal | Release Notes | 95

• Added Cylinder layer type to FHMDLayerDesc in HeadMountedDisplayCommon.h.

• In IInputInterface.h, FHapticFeedbackBuffer uint8* RawData was changed to TArray<uint8> RawData, and uint8 *CurrentPtr; was changed to uint32 CurrentPtr.

• IInputInterface::SetHapticFeedbackBuffer was removed and HapticBuffer was added to

FHapticFeedbackValues::FHapticFeedbackBuffer* HapticBuffer. You should now use

IInputInterface::SetHapticFeedbackValues, whether you are using a buffer or not.

• APlayerController::SetHatpicsByValue calls to Oculus Touch are now limited to 30 per second due to performance issues. Additional calls with a value other than zero (i.e., stop commands) are discarded upon receipt.

Known Issues

• UE4.12 and 4.13 in Oculus GitHub repository: A significant drop in frame rate occurs when UE4 is not in focus in VR preview mode. To avoid this issue, uncheck the Use Less CPU when in

Background in Edit >

Editor Preferences > General (left sidebar) > Miscellaneous (left sidebar) > Performance.

•

Stereo Layer Position Type: World Locked is not currently supported for cylinder TimeWarp overlays.

Oculus Unreal Engine 4 Integration 1.6 Release

Notes

1.6.0

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows.

Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

For more information on downloading and using the Oculus Unreal integration, please see

page 5.

The Wwise redirection patch is no longer required in Unreal Engine 4.11 or 4.12. Requires the WwiseUE4Plugin from AudioKinetic’s ‘master’ branch on GitHub.

4.12

New Features

• Added adaptive viewport scaling, which automatically changes pixel density based on resource availability, with configurable min/max. See

on page 48 for details. (Rift only)

UPDATE: This feature has been delayed until a future release.

• Added Blueprints plugin that supports calls for Gear VR functionality such as GPU/GPU throttling, querying headphone connection, querying device temperature, and more.

• Added Blueprint for Oculus Platform entitlement checks.

API Changes

• Oculus Touch: Added PlayHapticsSoundwave to Blueprints, which permits playback of soundwaves (stored in UE4 in USoundWave assets) directly to the Oculus Touch controller.

• Added SetHapticFeedbackBuffer to the C++ Haptics interface, to submit byte arrays to be played at 320 Hz on the Oculus Touch controller.

Bug Fixes

96 | Release Notes | Unreal

• Fixed bug where unfocused Wwise apps continued playing audio. Requires the WwiseUE4Plugin from

AudioKinetic’s ‘master’ branch on Github.

4.11

New Features

• Added adaptive viewport scaling, which automatically changes pixel density based on resource availability, with configurable min/max. See

on page 48 for details. (Rift only)

UPDATE: This feature has been delayed until a future release.

API Changes

• Oculus Touch: Added PlayHapticsSoundwave to Blueprints, which permits playback of soundwaves (stored in UE4 in USoundWave assets) directly to the Touch controller.

• Added SetHapticFeedbackBuffer to the C++ Haptics interface, to submit byte arrays to be played at 320 Hz on the Touch controller.

Bug Fixes

• Fixed bug where unfocused Wwise apps continued playing audio. Requires the WwiseUE4Plugin from

AudioKinetic’s ‘master’ branch on GitHub.

Oculus Unreal Engine 4 Integration 1.5 Release

Notes

1.5.0

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows.

Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

For more information on downloading and using the Oculus Unreal integration, please see

page 5.

4.12

New Features

• Added async splash (Gear VR).

• Refactored layers management (Gear VR).

• Enabled default loading icon mode (Gear VR).

• Added ability to capture cube maps for Oculus Store preview (see

on page 48 for usage instructions).

• Added call ovr_IdentifyClient to identify client to service.

• Added OnlineSubsystemOculus to interface Oculus platform.

API Changes

• Added GetNumOfTrackingSensors / GetTrackingSensorProperies to IHeadMountedDisplay.

Bug Fixes

• Fixed issue with BeginPlay.

Unreal | Release Notes | 97

• Fixed rounding error that could cause incorrect viewport sizes.

• Now use ovr_GetSessionStatus() instead of SubmitFrame result to determine VR focus.

• Fixed crash creating HMD when it is attached to D3D11 adapter with no other monitors.

• Fixed issue where IHeadMountedDisplayModule::IsHMDConnected would be called more than necessary.

4.11

New Features

• Added async splash (Gear VR).

• Refactored layers management (Gear VR).

• Enabled default loading icon mode (Gear VR).

• Added ability to capture cube maps for Oculus Store preview (see

on page 48 for usage instructions).

• Added ovr_IdentifyClient call to identify client to service.

API Changes

• Added GetNumOfTrackingSensors / GetTrackingSensorProperies to IHeadMountedDisplay.

Bug Fixes

• Now uses ovr_GetSessionStatus() instead of SubmitFrame result to determine VR focus.

• Fixed issue where IHeadMountedDisplayModule::IsHMDConnected would be called more than necessary.

Oculus Unreal Engine 4 Integration 1.4 Release

Notes

1.3.2

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows.

Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

For more information on downloading and using the Oculus Unreal integration, please see

page 5.

New Features

• Updated integration up to 4.11.2.

• Added Oculus Rift video layer support to 4.11 integration.

• Upgraded Oculus Mobile SDK to VrApi 1.0.2; Volume Control layer is now rendered correctly.

Bug Fixes

• Fixed potential Gear VR crash when EnterVRMode is called before RenderThread is started.

• Switched LOD calculation from view-vector dot product to pure distance for more consistent result

(otherwise, LOD could be different for each eye).

• Fixed incorrect logic for ATW splash rotation (PC).

98 | Release Notes | Unreal

Known Issues

• Mirror window may appear stretched with hmd mirror mode 2 until you switch focus to another application and back.

Oculus Unreal Engine 4 Integration 1.3 Release

Notes

1.3.2

Unreal provides built-in support for Oculus Rift and Gear VR development on Windows.

Unreal apps run on the Oculus platform automatically apply stereoscopic rendering to the main camera as well as positional and orientation tracking for the Rift, or orientation tracking for Gear VR.

For more information on downloading and using the Oculus Unreal integration, please see