Controller input and tracking is the most basic input format for any XR application. It enables users to perform complex interactions with the virtual world using a familiar interface. This guide will show you how to get basic controller input and tracking up and running in your Unreal Engine project. It will then demonstrate how to use the controller input and tracking to implement a basic interaction with objects in the world.

When you finish this guide, you should be able to:

Add Quest Controller handling to an Unreal Engine project.

Receive user input through a controller’s triggers.

Receive input when the user presses a controller’s thumbstick.

Provide haptic feedback.

Receive position and rotation information of a controller (6DOF).

Prerequisites

This guide relies on the Meta XR plugin for Unreal Engine. You can obtain it from the Unreal Marketplace. Please make sure you have it installed before continuing.

Project Setup

Your project must have the Meta XR plugin enabled. You will also need to have your project setup to deploy to a Meta Quest headset. Please make sure you have followed Creating Your First Meta Quest VR App in Unreal Engine before you proceed.

Implementation

Set up Game Framework

To begin adding your own functionality, you need to create and configure some custom framework classes and tell the application to use them instead of the defaults.

In the Content Drawer, create a new Blueprint Class asset using *Game Mode Base as the parent class. Name the new asset appropriately for your use case. In this example, it is called ControllerGameMode.

In the Content Drawer, create a new Blueprint Class asset using *Pawn as the parent class. Name the new asset appropriately for your use case. In this example, it is called ControllerPawn.

In the Window menu, select World Settings. Assign your new Game Mode asset to the GameMode Override property to use your Game Mode when you run the application.

In the Content Drawer, double-click the Game Mode asset to open it for editing. In the Blueprint Editor toolbar, click Class Defaults if it is not already selected. Assign your new Pawn asset to the Default Pawn Class property to force your Pawn class to be used with your Game Mode.

Add Input Mapping

To accept input from the user, you need to set up mappings that pass the hardware inputs through to your framework.

In the Content Drawer, add a new Input Action asset. This will be for the A button press oin the right controller so name it appropriately. In this example, it is called XR_R_A_InputAction.

Repeat this process for all additional inputs you want to have access to. At minimum, you will need Input Actions for A button press and Grip Axis for this guide. If you add an Input Action for an axis input, you need to double-click the asset and set the Value Type property to the correct type.

In the Content Drawer, add a new Input Mapping Context asset and name it appropriately. In this example, it is called XRInputMappingContext.

Double-click the Input Mapping Context asset to edit it and add entries to the Mappings array for each Input Action, associating each with the corresponding input.

Add mappings in Input Mappings Content asset

Set up Pawn Components

The custom Pawn class is where the controller tracking takes place. You need to add specialized components to this class to be able to access the controller data and visualize the controllers in the world.

In the Content Drawer, double-click the Pawn asset to open it for editing.

In the Components panel, add a new Motion Controller component under the DefaultSceneRoot component. Name the new component LeftMotionController.

With the LeftMotionController component selected, set the Motion Source property to Left in the Details panel.

In the Components panel, add a new Motion Controller component under the DefaultSceneRoot component. Name the new component RightMotionController.

With the RightMotionController component selected, set the Motion Source property to Right in the Details panel.

In the Components panel, add a new OculusXRController component under the LeftMotionController component. Name the component LeftOculusXRController.

With the LeftOculusXRController component selected, set the Skeleton Type property to Left in the Details panel.

In the Components panel, add a new OculusXRController component under the RightMotionController component. Name the component RightOculusXRController.

Add Right Oculus XR Controller component

With the RightOculusXRController component selected, set the Skeleton Type property to Right in the Details panel.

In the Components panel, add a new Camera component under the DefaultSceneRoot component. Name the component VRCamera.

Configure Tracking and Add Input Mapping

For tracking and input to work, you need to set up the tracking space and associate the Input Mapping Context you created previously with the player.

In the Event Graph, right-click by the Begin Play event and choose Is Head Mounted Display Enabled.

Add Is Head Mounted Display Enabled node

Drag off the output pin of the Is Head Mounted Display Enabled node and choose Branch.

Drag a connection from the output exec pin of the Begin Play event to the input exec pin of the Branch node.

Drag off the output exec pin of the Branch node and choose Set Tracking Origin.

On the Set Tracking Origin function, set the Origin pin value to Local Floor.

Right-click after the Set Tracking Origin function and choose Get Player Controller.

Drag off the output pin of the Get Player Controller node and choose Get EnahancedInputLocalPlayerSubsystem.

Add Get EnahancedInputLocalPlayerSubsystem node

Drag off the Enhanced Input Local Player Subsystem node and choose Add Mapping Context.

On the Add Mapping Context function, set the Mapping Context pin value to reference the Input Mapping Context asset you created previously.

Drag a connection from the output exec pin of the Set Tracking Origin function to the input exec pin of the Add Mapping Context function.

The completed Begin Play event network should look like this:

Access and Store Controller Tracking Data

Use the Tick event to access and store the controller tracking data each frame. You can use the stored data in other locations without having to execute the logic to access it over and over, which saves time during development and makes it more efficient at runtime.

Drag the RightOculusXRController component from the Components section of the My Blueprint panel to the Event Graph next to the Tick event. Select Get RightOculusXRController.

Drag off the output pin of the RightOculusXRController node and choose Get Motion Controller Data.

On the Get Motion Controller Data function, set the Hand pin value to Right.

Drag a connection from the output exec pin of the Tick event to the input exec pin of the Get Motion Controller Data function.

Drag off the Motion Controller Data output pin of the Get Motion Controller Data function and choose Promote to Variable.

In the Variables section of the My Blueprint panel, rename the new variable to Right Motion Controller Data.

Rename variable to Right Motion Controller Data

Drag off the output pin of the Set node and choose Break XRMotionControllerData.

Click the down arrow on the Break XRMotionControllerData node to expose all of the output pins.

Drag off the output exec pin of the Set node and choose Print String.

Drag a connection from the Aim Position output pin of the Break XRMotionControllerData node to the In String input pin of the Print String function.