Advanced VR Mechanics With Unity and the HTC Vive Part 1

VR is more popular than ever, and making games has never been easier. But to offer a really immersive experience, your in-game mechanics and physics need to feel very, very real, especially when you’re interacting with in-game objects.

In the first part of this advanced HTC Vive tutorial, you’ll learn how to create an expandable interaction system and implement multiple ways to grab virtual objects inside that system, and fling them around like nobody’s business.

By the time you’re done, you’ll have some flexible interaction systems that you can use right in your own VR projects!

Getting Started

You’ll need the following things for this tutorial:

• A copy of Unity 5.6.0f3 (or newer) installed on your machine.
• A HTC Vive with controllers that are set up, powered on, and ready to go.

If you haven’t worked with the HTC Vive before, you might want to check out this previous HTC Vive tutorial to get a feel for the basics of working with the HTC Vive in Unity. The HTC Vive is one of the best head-mounted displays at the moment and offers an excellent immersive experience because of its room-scale gameplay capabilities.

Download the starter project, unzip it somewhere and open the folder inside Unity.

Take a look at the folder structure in the Project window:

Here’s what each contains:

• Materials: All the materials for the scene.
• Models: All models for this tutorial.
• Prefabs: For now, this only contains the prefab for the poles that are scattered around the level. You’ll place your own objects in here for later use.
• Scenes: The game scene and some lighting data.
• Scripts: A few premade scripts; you’ll save your own scripts in here as well.
• Sounds: The sound for shooting an arrow from the bow.
• SteamVR: The SteamVR plugin and all related scripts, prefabs and examples.
• Textures: Contains the main texture shared by almost all models (for the sake of efficiency) as well as the texture for the book object.

Open up the Game scene inside the Scenes folder.

Look at the Game view and you’ll notice there’s no camera present in the scene:

In the next section you’ll fix this by adding everything necessary for the HTC Vive to work.

Scene Setup

Select and drag the [CameraRig] and [SteamVR] prefabs from the SteamVR\Prefabs folder to the Hierarchy.

The camera rig will now be on the ground, but it should be on the wooden tower. Change the position of [CameraRig] to (X:0, Y:3.35, Z:0) to correct this. This is what it should look like in the Game view:

Now save the scene and press the play button to test if everything works as intended. Be sure to look around and use at least one controller to see if you can see the in-game controller moving around.

If the controllers didn’t work, don’t panic! At the time of writing, there’s a bug in the latest SteamVR plugin (version 1.2.1) when using Unity 5.6 which causes the movement of the controllers to not register.

To fix this, select Camera (eye) under [CameraRig]/Camera (head) and add the SteamVR_Update_Poses component to it:

This script manually updates the position and rotation of the controllers. Try playing the scene again, and things should work much better.

Before doing any scripting, take a look at these tags in the project:

These tags make it easier to detect which type of object collided or triggered with another.

Interaction System: InteractionObject

An interaction system allows for a flexible, modular approach to interactions between the player and objects in the scene. Instead of rewriting the boilerplate code for every object and the controllers, you’ll be making some classes from which other scripts can be derived.

The first script you’ll be making is the RWVR_InteractionObject class; all objects that can be interacted with will be derived from this class. This base class will hold some essential variables and methods.

Create a new folder in the Scripts folder and name it RWVR. Create a new C# script in there and name it RWVR_InteractionObject.

Open up the script in your favorite code editor and remove both the Start() and Update() methods.

Add the following variables to the top of the script, right underneath the class declaration:

protected Transform cachedTransform; // 1
[HideInInspector] // 2
public  RWVR_InteractionController currentController; // 3

You’ll probably get an error saying RWVR_InteractionController couldn’t be found. Ignore this for now, as you’ll be creating that class next.

Taking each commented line in turn:

1. You cache the value of the transform to improve performance.
2. This attribute makes the variable underneath invisible in the Inspector window, even though it’s public.
3. This is the controller this object is currently interacting with. You’ll visit the controller in detail later on.

Save this script for now and return to the editor.

Create a new C# script inside the RWVR folder named RWVR_InteractionController. Open it up, remove the Start() and Update() methods and save your work.

Open the RWVR_InteractionObject script again, and the error you received before should be gone.

Now add the following three methods below the variables you just added:

public virtual void OnTriggerWasPressed(RWVR_InteractionController controller)
{
    currentController = controller; 
}

public virtual void OnTriggerIsBeingPressed(RWVR_InteractionController controller)
{
}

public virtual void OnTriggerWasReleased(RWVR_InteractionController controller)
{
    currentController = null;
}

These methods will be called by the controller when its trigger is either pressed, held or released. A reference to the controller is stored when it’s pressed, and removed again when it’s released.

All of these methods are virtual and will be overridden by more sophisticated scripts later on so they can benefit from these controller callbacks.

Add the following method below OnTriggerWasReleased:

public virtual void Awake()
{
    cachedTransform = transform; // 1
    if (!gameObject.CompareTag("InteractionObject")) // 2
    {
        Debug.LogWarning("This InteractionObject does not have the correct tag, setting it now.", gameObject); // 3
        gameObject.tag = "InteractionObject"; // 4
    }
}

Taking it comment-by-comment:

1. Cache the transform for better performance.
2. Check to see if this InteractionObject has the proper tag assigned. Execute the code below if it doesn’t.
3. Log a warning in the inspector to warn the developer of a forgotten tag.
4. Assign the tag just in time so this object functions as expected.

The interaction system will depend heavily upon the InteractionObject and Controller tags to differentiate those special objects from the rest of the scene. It’s quite easy to forget to add this tag to objects every time you add a script to it. That’s why this failsafe is in place. Better to be safe than sorry! :]

Finally, add these methods below Awake():

public bool IsFree() // 1
{
    return currentController == null;
}

public virtual void OnDestroy() // 2
{
    if (currentController)
    {
        OnTriggerWasReleased(currentController);
    }
}

Here’s what these methods do:

1. This is a public Boolean that indicates whether or not this object is currently in use by a controller.
2. When this object gets destroyed, you release it from the current controller (if there are any). This helps to avoid weird bugs later on when working with objects that can be held.

Save this script and open the RWVR_InteractionController script again.

It’s empty at the moment. But you’ll soon fill it up with functionality!

Interaction System: Controller

The controller script might be the most important piece of all, as it’s the direct link between the player and the game. It’s important to make use of as much input as possible and return appropriate feedback to the player.

To start off, add the following variables below the class declaration:

public Transform snapColliderOrigin; // 1
public GameObject ControllerModel; // 2

[HideInInspector]
public Vector3 velocity; // 3
[HideInInspector]
public Vector3 angularVelocity; // 4

private RWVR_InteractionObject objectBeingInteractedWith; // 5

private SteamVR_TrackedObject trackedObj; // 6

Looking at each piece in turn:

1. Save a reference to the tip of the controller. You’ll be adding a transparent sphere later, which will act as a guide to where and how far you can reach:

   

2. This is the visual representation of the controller, seen in white above.
3. This is the speed and direction of the controller. You’ll use this to calculate how objects should fly when you throw them.
4. This is the rotation of the controller, also used when calculating the motion of thrown objects.
5. This is the InteractionObject this controller is currently interacting with. You use it to send events to the active object.
6. SteamVR_TrackedObject can be used to get a reference to the actual controller.

Add this code below the variables you just added:

private SteamVR_Controller.Device Controller // 1
{
    get { return SteamVR_Controller.Input((int)trackedObj.index); }
}

public RWVR_InteractionObject InteractionObject // 2
{
    get { return objectBeingInteractedWith; }
}

void Awake() // 3
{
    trackedObj = GetComponent<SteamVR_TrackedObject>();
}

Here’s what’s going on in the code above:

1. This variable acts as a handy shortcut to the actual SteamVR controller class from the tracked object.
2. This returns the InteractionObject this controller is currently interacting with. It’s been encapsulated to ensure it stays read-only for other classes.
3. Finally, save a reference to the TrackedObject component attached to this controller to use later.

Now add the following method:

private void CheckForInteractionObject()
{
    Collider[] overlappedColliders = Physics.OverlapSphere(snapColliderOrigin.position, snapColliderOrigin.lossyScale.x / 2f); // 1

    foreach (Collider overlappedCollider in overlappedColliders) // 2
    {
        if (overlappedCollider.CompareTag("InteractionObject") && overlappedCollider.GetComponent<RWVR_InteractionObject>().IsFree()) // 3
        {
            objectBeingInteractedWith = overlappedCollider.GetComponent<RWVR_InteractionObject>(); // 4
            objectBeingInteractedWith.OnTriggerWasPressed(this); // 5
            return; // 6
        }
    }
}

This method searches for InteractionObjects in a certain range from the controller’s snap collider. Once it finds one, it populates the objectBeingInteractedWith with a reference to it.

Here’s what each line does:

1. Creates a new array of colliders and fills it with all colliders found by OverlapSphere() at the position and scale of the snapColliderOrigin, which is the transparent sphere shown above that you’ll add shortly.
2. Iterates over the array.
3. If any of the found colliders has an InteractionObject tag and is free, continue.
4. Saves a reference to the RWVR_InteractionObject attached to the object that was overlapped in objectBeingInteractedWith.
5. Calls OnTriggerWasPressed on objectBeingInteractedWith and gives it the current controller as a parameter.
6. Breaks out of the loop once an InteractionObject is found.

Add the following method that makes use of the code you just added:

void Update()
{
    if (Controller.GetHairTriggerDown()) // 1
    {
        CheckForInteractionObject();
    }

    if (Controller.GetHairTrigger()) // 2
    {
        if (objectBeingInteractedWith)
        {
            objectBeingInteractedWith.OnTriggerIsBeingPressed(this);
        }
    }

    if (Controller.GetHairTriggerUp()) // 3
    {
        if (objectBeingInteractedWith)
        {
            objectBeingInteractedWith.OnTriggerWasReleased(this);
            objectBeingInteractedWith = null;
        }
    }
}

This is fairly straightforward:

1. When the trigger is pressed, call CheckForInteractionObject() to prepare for a possible interaction.
2. While the trigger is held down and there’s an object being interacted with, call the object’s OnTriggerIsBeingPressed().
3. When the trigger is released and there’s an object that’s being interacted with, call that object’s OnTriggerWasReleased() and stop interacting with it.

These checks make sure that all of the player’s input is being passed to any InteractionObjects they are interacting with.

Add these two methods to keep track of the controller’s velocity and angular velocity:

private void UpdateVelocity()
{
    velocity = Controller.velocity;
    angularVelocity = Controller.angularVelocity;
}

void FixedUpdate()
{
    UpdateVelocity();
}

FixedUpdate() calls UpdateVelocity() every frame at the fixed framerate, which updates the velocity and angularVelocity variables. Later, you’ll pass these values to a RigidBody to make thrown objects move more realistically.

Sometimes you’ll want to hide a controller to make the experience more immersive and avoid blocking your view. Add the following two methods below the previous ones:

public void HideControllerModel()
{
    ControllerModel.SetActive(false);
}

public void ShowControllerModel()
{
    ControllerModel.SetActive(true);
}

These methods simply enable or disable the GameObject representing the controller.

Finally, add the following two methods:

public void Vibrate(ushort strength) // 1
{
    Controller.TriggerHapticPulse(strength);
}

public void SwitchInteractionObjectTo(RWVR_InteractionObject interactionObject) // 2
{
    objectBeingInteractedWith = interactionObject; // 3
    objectBeingInteractedWith.OnTriggerWasPressed(this); // 4
}

Here’s how these methods work:

1. This method makes the piezoelectric linear actuators (no, I’m not making that up) inside the controller vibrate for a certain amount of time. The longer it vibrates, the stronger the vibration feels. Its range is between 1 and 3999.
2. This switches the active InteractionObject to the one specified in the parameter.
3. This makes the specified InteractionObject the active one.
4. Call OnTriggerWasPressed() on the newly assigned InteractionObject and pass this controller.

Save this script and return to the editor. In order to get the controllers working as intended, you’ll need to make a few adjustments.

Select both controllers in the Hierarchy. They’re both children of [CameraRig].

Add a Rigidbody component to both. This will allow them to work with fixed joints and interact with other physics objects.

Uncheck Use Gravity and check Is Kinematic. The controllers don’t need be to affected by physics since they’re strapped to your hands in real life.

Now add the RWVR_Interaction Controller component to the controllers. You’ll configure those in a bit.

Unfold Controller (left) and add a Sphere to it as its child by right-clicking it and selecting 3D Object > Sphere.

Select Sphere, name it SnapOrigin and press F to focus on it in the Scene view. You should see a big white hemisphere at the center of the platform floor:

Set its Position to (X:0, Y:-0.045, Z:0.001) and its Scale to (X:0.1, Y:0.1, Z:0.1). This will position the sphere right at the front of the controller.

Remove the Sphere Collider component, as all physics checks are done in code.

Finally, make the sphere transparent by applying the Transparent material to its Mesh Renderer.

Now duplicate SnapOrigin and drag SnapOrigin (1) to Controller (right) to make it a child of the right controller. Name it SnapOrigin.

The final step is to set up the controllers to make use of their Model and SnapOrigin.

Select and unfold Controller (left), drag its child SnapOrigin to the Snap Collider Origin slot and drag Model to the Controller Model slot.

Do the same for Controller (right).

Now for a bit of fun! Power on your controllers and run the scene.

Move the controllers in front of the HMD to check if the spheres are clearly visible and attached to the controllers.

When you’re done testing, save the scene and prepare to actually use the interaction system!

Grabbing Objects Using The Interaction System

You may have noticed these objects laying around:

You can take a good look at them, but you can’t pick them up yet. You’d better fix that soon, or how will you ever learn how awesome our Unity book is?! :]

In order to interact with rigidbodies like these, you’ll need to create a new derivative class of RWVR_InteractionObject that will let you grab and throw objects.

Create a new C# script in the Scripts/RWVR folder and name it RWVR_SimpleGrab.

Open it up in your code editor and remove the Start() and Update() methods.

Replace the following:

public class RWVR_SimpleGrab : MonoBehaviour

…with:

public class RWVR_SimpleGrab : RWVR_InteractionObject

This makes this script derive from RWVR_InteractionObject, which provides all the hooks onto the controller’s input so it can appropriately handle the input.

Add these variables below the class declaration:

public bool hideControllerModelOnGrab; // 1
private Rigidbody rb; // 2

Quite simply:

1. A flag indicating whether or not the controller model should be hidden when this object is picked up.
2. Cache the Rigidbody component for performance and ease of use.

Add the following methods below those variables:

public override void Awake()
{
    base.Awake(); // 1
    rb = GetComponent<Rigidbody>(); // 2
}

Short and sweet:

1. Call Awake() on the base class RWVR_InteractionObject. This caches the object’s Transform component and checks if the InteractionObject tag is assigned.
2. Store the attached Rigidbody component for later use.

Now you need some helper methods that will attach and release the object to and from the controller by using a FixedJoint.

Add the following methods below Awake():

private void AddFixedJointToController(RWVR_InteractionController controller) // 1
{
    FixedJoint fx = controller.gameObject.AddComponent<FixedJoint>();
    fx.breakForce = 20000;
    fx.breakTorque = 20000;
    fx.connectedBody = rb;
}

private void RemoveFixedJointFromController(RWVR_InteractionController controller) // 2
{
    if (controller.gameObject.GetComponent<FixedJoint>())
    {
        FixedJoint fx = controller.gameObject.GetComponent<FixedJoint>();
        fx.connectedBody = null;
        Destroy(fx);
    }
}

Here’s what these methods do:

1. This method accepts a controller to “stick” to as a parameter and then proceeds to create a FixedJoint component. Attach it to the controller, configure it so it won’t break easily and finally connect it to the current InteractionObject. The reason you set a finite break force is to prevent users from moving objects through other solid objects, which might result in weird physics glitches.
2. The controller passed as a parameter is relieved from its FixedJoint component (if there is one). The connection to this object is removed and the FixedJoint is destroyed.

With those methods in place, you can take care of the actual player input by implementing some OnTrigger methods from the base class. To start off with, add OnTriggerWasPressed():

public override void OnTriggerWasPressed(RWVR_InteractionController controller) // 1
{
    base.OnTriggerWasPressed(controller); // 2

    if (hideControllerModelOnGrab) // 3
    {
        controller.HideControllerModel();
    }

    AddFixedJointToController(controller); // 4
}

This method adds the FixedJoint when the player presses the trigger button to interact with the object. Here’s what you do in each part:

1. Override the base OnTriggerWasPressed() method.
2. Call the base method to intialize the controller.
3. If the hideControllerModelOnGrab flag was set, hide the controller model.
4. Add a FixedJoint to the controller.

The final step for this simple grab class is to add OnTriggerWasReleased():

public override void OnTriggerWasReleased(RWVR_InteractionController controller) // 1
{
    base.OnTriggerWasReleased(controller); //2

    if (hideControllerModelOnGrab) // 3
    {
        controller.ShowControllerModel();
    }

    rb.velocity = controller.velocity; // 4
    rb.angularVelocity = controller.angularVelocity;

    RemoveFixedJointFromController(controller); // 5
}

This method removes the FixedJoint and passes the controller’s velocities to create a realistic throwing effect. Comment-by-comment:

1. Override the base OnTriggerWasReleased() method.
2. Call the base method to unassign the controller.
3. If the hideControllerModelOnGrab flag was set, show the controller model again.
4. Pass the controller’s velocity and angular velocity to this object’s rigidbody. This means the object will react in a realistic manner when you release. For example, if you’re throwing a ball, you move the controller from back-to-front in an arc. The ball should gain rotation and a forward-acting force as if you had passed your actual kinetic energy in real life.
5. Remove the FixedJoint.

Save this script and return to the editor.

The dice and books are linked to their respective prefabs in the Prefabs folder. Open this folder in the Project view:

Select the Book and Die prefabs and add the RWVR_Simple Grab component to both. Also enable Hide Controller Model.

Save and run the scene. Try grabbing some of the books and dice and throwing them around.

In the next section I’ll explain another way of grabbing objects: via snapping.

Grabbing and Snapping Objects

Grabbing objects at the position and rotation of your controller usually works, but in some cases snapping the object to a certain position might be desirable. For example, when the player sees a gun, they would expect the gun to be pointing in the right direction once they’ve picked it up. This is where snapping comes into play.

In order for snapping to work, you’ll need to create another script. Create a new C# script inside the Scripts/RWVR folder and name it RWVR_SnapToController. Open it in your favorite code editor and remove the Start() and Update() methods.

Replace the following:

public class RWVR_SnapToController : MonoBehaviour

..with:

public class RWVR_SnapToController : RWVR_InteractionObject

This lets this script use all of the InteractionObject capabilities.

Add the following variable declarations:

public bool hideControllerModel; // 1
public Vector3 snapPositionOffset; // 2
public Vector3 snapRotationOffset; // 3

private Rigidbody rb; // 4

Here’s what these variables are for:

1. A flag to tell whether the controller’s model should be hidden once the player grabs this object.
2. The position added after snapping. The object snaps to the controller’s position by default.
3. Same as above, except this handles the rotation.
4. A cached reference of this object’s Rigidbody component.

Add the following method below the variables:

public override void Awake()
{
    base.Awake();
    rb = GetComponent<Rigidbody>();
}

Just as in the SimpleGrab script, this overrides the base Awake() method, calls the base and caches the RigidBody component.

Next up are the helper methods, which form the real meat of this script.

Add the following method below Awake():

private void ConnectToController(RWVR_InteractionController controller) // 1
{
    cachedTransform.SetParent(controller.transform); // 2

    cachedTransform.rotation = controller.transform.rotation; // 3
    cachedTransform.Rotate(snapRotationOffset);
    cachedTransform.position = controller.snapColliderOrigin.position; // 4
    cachedTransform.Translate(snapPositionOffset, Space.Self);

    rb.useGravity = false; // 5
    rb.isKinematic = true; // 6
}

The way this script attaches the object differs from the SimpleGrab script, as it doesn’t use a FixedJoint, but instead makes itself a child of the controller. This means the connection between the controller and snap objects can’t be broken by force. This will keep everything stable for this tutorial, but you might prefer to use a FixedJoint in your own projects.

Taking it play-by-play:

1. Accept a controller as a parameter to connect to.
2. Set this object’s parent to be the controller.
3. Make this object’s rotation the same as the controller and add the offset.
4. Make this object’s position the same as the controller and add the offset.
5. Disable the gravity on this object; otherwise, it would fall out of your hand.
6. Make this object kinematic. While attached to the controller, this object won’t be under the influence of the physics engine.

Now add the matching method to release the object by adding the following method:

private void ReleaseFromController(RWVR_InteractionController controller) // 1
{
    cachedTransform.SetParent(null); // 2

    rb.useGravity = true; // 3
    rb.isKinematic = false;

    rb.velocity = controller.velocity; // 4
    rb.angularVelocity = controller.angularVelocity;
}

This simply unparents the object, resets the rigidbody and applies the controller velocities. In more detail:

1. Accept the controller to release as a parameter.
2. Unparent the object.
3. Re-enable gravity and make the object non-kinematic again.
4. Apply the controller’s velocities to this object.

Add the following override method to perform the snapping:

public override void OnTriggerWasPressed(RWVR_InteractionController controller) // 1
{
    base.OnTriggerWasPressed(controller); // 2

    if (hideControllerModel) // 3
    {
        controller.HideControllerModel();
    }

    ConnectToController(controller); // 4
}

This one is fairly straightforward:

1. Override OnTriggerWasPressed() to add the snap code.
2. Call the base method.
3. If the hideControllerModel flag was set, hide the controller model.
4. Connect this object to the controller.

Now add the release method below:

public override void OnTriggerWasReleased(RWVR_InteractionController controller) // 1
{
    base.OnTriggerWasReleased(controller); // 2

    if (hideControllerModel) // 3
    {
        controller.ShowControllerModel();
    }

    ReleaseFromController(controller); // 4
}

Again, fairly simple:

1. Override OnTriggerWasReleased() to add the release code.
2. Call the base method.
3. If the hideControllerModel flag was set, show the controller model again.
4. Release this object to the controller.

Save this script and return to the editor. Drag the RealArrow prefab out of the Prefabs folder into the Hierarchy window.

Select the arrow and set its position to (X:0.5, Y:4.5, Z:-0.8). It should be floating above the stone slab now:

Attach the RWVR_Snap To Controller component to the new arrow in the Hierarchy so you can interact with it and set its Hide Controller Model bool to true. Finally, press the Apply button at the top of the Inspector window to apply the changes to this prefab.

For this object, there’s no need to change the offsets; it should snap to an acceptable position by default.

Save the scene and run it. Grab the arrow and throw it away. Let your inner beast out!

Notice that the arrow will always be positioned properly in your hand, no matter how you pick it up.

You’re all done with this tutorial; play around with the game a bit to get a feel for the dynamics of the interactions.

Where to Go From Here?

You can download the finished project here.

In this tutorial you’ve learned how to create an expandable interaction system, and you’ve discovered several ways to grab objects using the interaction system.

In the next part of this tutorial, you’ll learn how to expand the system further by making a functional bow and arrow, and even creating a functional backpack!

If you’re interested in learning more about creating killer games with Unity, check out our book, Unity Games By Tutorials.

In this book, you create four complete games from scratch:

• A twin-stick shooter
• A first-person shooter
• A tower defense game (with VR support!)
• A 2D platformer

By the end of this book, you’ll be ready to make your own games for Windows, macOS, iOS, and more!

This book is for complete beginners to Unity, as well as for those who’d like to bring their Unity skills to a professional level. The book assumes you have some prior programming experience (in a language of your choice).

Thanks for reading! If you have any comments or suggestions, please join the discussion below.