- Hololens Unity Toolkit Software
- Hololens Visual Studio Unity
- Hololens Unity Setup
- Hololens Unity Development
- Deploy From Unity To Hololens
by Max Huddleston
Microsoft’s HoloLens is pretty freaking awesome. I was introduced to it at my internship this summer, and I’ve had a blast developing for it.
HoloLens apps are created using either DirectX with C++ or Unity with C#. I found that it is generally quicker and easier to get an app up and running with Unity.
The HoloLens 2 tutorials are designed to help developers learn both techniques and best practices for developing mixed reality applications. The Unity tutorials are based on the Mixed Reality Toolkit 2.0 (MRTK 2.0), and the Unreal tutorial is based on the UX Tools for Unreal Engine 0.8.
In this article, we’ll set up Unity for HoloLens development, create an interactive cube, and spin up the HoloLens emulator so you can see your creation. This article is meant for total beginners to Unity, so if you have experience with the editor, I’d recommend going through Microsoft’s tutorials.
- In Unity, right click Assets folder in your Project Window Import Package Custom Package and select each unity package file downloaded, the import During import or under Mixed Reality Toolkit.
- A HoloLens 2 device (or you can use the emulator) Unity Hub with Unity 2019.3.X installed and the Universal Windows Platform (UWP). Again you can tick this module (UWP) during the installation process) Unity Mixed reality Tool Kit (MRTK) version 2.3.0 — Download from here. Let’s get started.
Prerequisites
- A Windows 10 PC with the April 2018 update
- The HoloLens Emulator which you can download here
- Visual Studio 2017 — Community edition is fine
- Unity — Make sure you add the Windows .NET scripting backend component during installation
- The HoloToolKit
You can find the full source code here.
Set up
Open up Unity. If this is your first time using the editor, you will be prompted to sign in or sign up. I’d recommend Unity Personal unless you plan on monetizing your app.
After you sign in, create a new project and enter in a name. You should be greeted with this screen after the project completes the initial setup.
Right click on the Assets folder in the bottom left corner. Navigate to Import Package -> Custom Package, and open the HoloToolKit unity file we downloaded earlier. A Unity popup will appear, click All and import the assets.
Next we’ll create our scene. Select File -> New Scene and save it as Main.
Unity needs a special configuration in order to build a project for HoloLens. Luckily, the HoloToolKit has utility scripts which do this configuration for us. On the top toolbar, select Mixed Reality Toolkit -> Configure -> Apply Mixed Reality Project Settings. Use the default values. After the configuration is finished, apply the Mixed Reality Scene Settings. Delete the Directional Light GameObject.
Awesome! So far we’ve imported the HoloToolKit and set up Unity for HoloLens development. In the next section I’ll introduce you to the basics of Unity, and after that we’ll get a cube to render on screen.
Familiarizing yourself with the editor
This is what the editor should look like at this point. Notice the three boxed-in areas:
The red area to the left is the scene hierarchy panel. Here we can add new GameObjects, and get a high-level overview of the current application.A GameObject is a base class for all Unity entities. They can be a physical object like a cube, or a collection of scripts like the Input Manager in the scene.
The bottom blue area contains the assets menu and console. The assets menu is exactly what it sounds like — it contains the C# scripts, materials, prefabs, and meshes required to run your app.
The right green area is the inspector menu. Here is where you can add and modify the Components of your GameObjects. Components are what make up the behavior of your GameObjects. They determine the look, interactivity, and physics of their parent.
Creating a cube
Now we’ll get our first GameObject to render on screen. In the scene hierarchy panel, right click and select 3D Object -> Cube. A cube should appear in the editor.
Double click Cube in the in hierarchy panel, and in the right inspector panel change the position to (0,0,2). Set the scale to be (.25, .25, .25). Your inspector panel should look like this:
Nice! Let’s go through the build steps so we can see our cube in the HoloLens emulator.
In Unity, select File->Build Settings. The build popup should appear, so hit the build button. In the folder popup, create a new folder called App and choose it as the build destination.
After the build is completed, open the Visual Studio solution in the App folder. Change the debug options to Release, x86, and target the HoloLens emulator. Your options bar should look like this:
Run the solution, and after the emulator starts up and your application loads, you should see your cube on screen.
Making the cube interactive
Now that we’ve figured out how to get a cube into our application, let’s make the cube actually do something. When we gaze at the cube, it will rotate, and when we click the cube, it will increase in size.
Navigate back to Unity and create a new script in the assets panel called InteractiveCube. To create a new script, right click on the assets panel and select Create -> C# script.
To add the script to the cube, make sure the cube is selected, and drag and drop the script onto the inspector panel. It should look like this:
Double click on the script in the assets tab and a Visual Studio instance should appear.
Import HoloToolkit.Unity.InputModule at the top of our script, and make InteractiveCube extend IFocusable and IInputClickHandler. Our code should look like this:
By extending the IFocuable and IInputClickHandler interfaces, our script component allows the parent GameObject to subscribe to focus and click events.
Let’s make the cube rotate when our gaze is on it. The IFocusable interface requires us to implement two public void methods: OnFocusEnter and OnFocusExit. Create a private boolean field and name it Rotating. When we focus the cube, set it to true, and when our focus exits, set it to false. Our code should look like this:
We’ll do the actual rotation in Update(). Update() is a special Unity method that is called every frame. To control the speed of the rotation, add a public float field named RotationSpeed. Any public field in a component can be adjusted and initialized in the Unity editor.
In Unity, the transform is used to control the physical attributes such as size, rotation, and position of a GameObject. We are rotating the parent GameObject around the y-axis at one degree per second multiplied by the speed.
To make the cube respond to click events, add the OnInputClicked method required by IInputClickHandler. Create a public Vector3 variable called ScaleChange. In the OnInputClicked method, we’ll increase the cube’s scale by ScaleChange.
Now that we’re finished with the script, navigate back to Unity. Make sure the Cube object is selected, and set the Speed and ScaleChange variables to 50 and (.025, .025, .025) respectively. Feel free to experiment with different values! Our script should look like this:
Awesome! We’ve finished the InteractiveCube script. Build your app from the Unity editor, and run the solution from Visual Studio.
Summary
In this walkthrough, you learned how to set up Unity for HoloLens development, create an interactive GameObject, and run your application in the emulator.
If you enjoy HoloLens development, I encourage you to go through the Microsoft Academy tutorials. They go over the various HoloLens key concepts in detail and walk you through creating some pretty cool apps.
If you liked the article or have feedback, leave a comment below!
-->Mixed Reality is a big place, a nexus where real and virtual worlds collide with a bang. The possibilities are endless, and it starts with you. Choose your engine, download your tools, and dust off your keyboard. Welcome to the Mixed Reality ecosystem - it's time to create.
Development options
Unity - Unity is one of the leading real-time development platforms on the market, with underlying runtime code written in C++ and all development scripting is done in C#. Whether you're looking to build games, movies and animation cinematics, or even render architectural or engineering concepts in a virtual world, Unity has the infrastructure to support you.
Unreal - Unreal Engine 4 is a powerful, open source creation engine with full support for mixed reality in both C++ and Blueprints. As of Unreal Engine 4.25, HoloLens support is full-featured and production-ready. With capabilities such as the flexible Blueprints Visual Scripting system, designers can virtually use the full range of concepts and tools generally only available to programmers. Creators across industries can leverage the freedom and control to deliver cutting-edge content, interactive experiences, and immersive virtual worlds.
Native (OpenXR) - OpenXR is an open royalty-free API standard from Khronos that provides engines native access to a wide range of devices from vendors across the mixed reality spectrum. You can develop using OpenXR on a HoloLens 2 or Windows Mixed Reality immersive headset on the desktop. If you don't have access to a headset, emulators for HoloLens 2 and Windows Mixed Reality headsets are available.
What technology path are you interested in?
There are several development paths you can take through our documentation. The first step is finding the technology that's right for you. If you already have one in mind, go ahead and jump right to its respective tab below. If you're on the fence or just starting out, take a look through each one and understand what they offer, the available platforms and tools, and start creating!
Important
Take a look at our porting guides overview if you have existing projects that you want to bring over to HoloLens 2 or immmersive VR headsets like the Reverb G2. We have guides for projects that are using HTK, MRTK v1, SteamVR or were developed for immersive headsets such as the Oculus Rift or HTC Vive.
Build a cross-platform, full-featured mixed reality app with Unity. See the Unity development overview to get started with Unity development for HoloLens or Windows Mixed Reality immersive headsets.
Available hardware platforms
You have several hardware and emulator options when building Mixed Reality apps with Unity. While our developer documentation focuses on HoloLens devices, you'll find device support sections with details on immersive headset deployment when applicable.
Augmented reality devices
Immersive VR headsets
- HP Reverb and Reverb G2
- Samsung Odyssey and Odyssey+
- HP Windows Mixed Reality headset
- Lenovo Explorer
- Acer AH101
- Dell Visor
- Asus HC102
- Acer OJO 500
Available tools and SDKs
Tool/SDK | Description |
---|---|
Mixed Reality Toolkit for Unity | The Mixed Reality Toolkit for Unity is an open source cross-platform development kit built to accelerate development of applications targeting Microsoft HoloLens, Windows Mixed Reality immersive (VR) headsets and OpenVR platform. |
Cloud services
There are several cloud services that can be integrated into Mixed Reality projects built in Unity, specifically Azure Remote Rendering and Azure Spatial Anchors. These services can add shared holographic content and real-time 3D rendering to your applications, making them more compelling and immersive for your users.
All of these services are covered over the course of the Unity development journey, which is the strongly recommended path for learning Mixed Reality with Unity. You're already on this path, so just keep reading and follow the big blue button at the bottom of the article. However, if you're at a more advanced development stage and already know you want to jump in, check out our cloud services overview or go directly to the services resources.
Dynamics 365 Guides
You can use Microsoft Dynamics 365 Guides to visually tether holographic instructions to your apps virtual environment, giving your users important information when and where it's needed. This feature is also covered in the Unity development journey, but if you want to jump ahead you can see what's on offer by selecting the Dynamics 365 tab here.
Examples
We have several open source sample apps for you to download and play around with to get a feel for a Mixed Reality end product in Unity. There are also MRTK example scenes available for you to test out specific features:
- Hand interaction examples scene (MRTK) for Unity -The HandInteractionExamples.unity example scene contains various types of interactions and UI controls that highlight articulated hand input.
- Eye tracking examples (MRTK) for Unity -This page covers how to get quickly started with using eye tracking in MRTK by building on our provided MRTK eye tracking examples.
Note
Both of the MRTK example scenes require MRTK Foundation and Example Unity packages to be installed.
Build a cross-platform, full-featured mixed reality app with Unreal. See the Unreal development overview to get started with Unreal development for HoloLens.
Available hardware platforms
Hololens Unity Toolkit Software
You have several hardware, emulator, and streaming options when building Mixed Reality apps with Unreal Engine. While our developer documentation focuses on HoloLens devices, you can package your Unreal projects as x64 desktop apps and run them on immersive headsets just fine.
Augmented reality devices
Immersive VR headsets
- HP Reverb and Reverb G2
- Samsung Odyssey and Odyssey+
- HP Windows Mixed Reality headset
- Lenovo Explorer
- Acer AH101
- Dell Visor
- Asus HC102
- Acer OJO 500
Available tools and SDKs
Tool/SDK | Description |
---|---|
Mixed Reality Toolkit for Unreal | The Mixed Reality Toolkit for Unreal (MRTK-Unreal) is a set of components, in the form of plugins, samples and documentation, designed to accelerate development of mixed reality applications using the Unreal Engine. |
Cloud services
When building Mixed Reality apps in Unreal, you have access to a powerful cloud service called Azure Spatial Anchors, which you can use to add, persist, and share holographic content across different devices.
Azure Spatial Anchors is covered over the course of the Unreal development journey, which is the strongly recommended path for learning Mixed Reality with Unreal. You're already on this path, so just keep reading and follow the big blue button at the bottom of the article. However, if you're at a more advanced development stage and already know you want to jump in, check out our cloud services overview or go directly to the services resources.
Dynamics 365 Guides
You can use Microsoft Dynamics 365 Guides to visually tether holographic instructions to your apps virtual environment, giving your users important information when and where it's needed. This feature is also covered in the Unreal development journey, but if you want to jump ahead you can see what's on offer by selecting the Dynamics 365 tab here.
Examples
Hololens Visual Studio Unity
- Kippy's Escape - Kippy’s Escape is an open-source HoloLens 2 sample app built with Unreal Engine 4 and Mixed Reality UX Tools for Unreal. The game showcases the unique features of the HoloLens 2 hardware and the development power of the Mixed Reality UX Tools.
The WebXR Device API is an open specification that allows you to experience Mixed Reality apps in your browser on any platform. See the JavaScript development overview to get started building Mixed Reality apps for any platform.
Create mixed reality apps with a direct line to the Windows Mixed Reality APIs. See the Native development overview to get started with native app development using OpenXR or legacy WinRT for HoloLens 2 or Windows Mixed Reality immersive headsets. The Windows Mixed Reality API supports applications written in C++ and C#, allowing you to build your own framework or middleware in either language.
Available hardware platforms
You have several hardware, emulator, and streaming options when building Mixed Reality apps with OpenXR development.
Augmented reality devices
Immersive VR headsets
- HP Reverb and Reverb G2
- Samsung Odyssey and Odyssey+
- HP Windows Mixed Reality headset
- Lenovo Explorer
- Acer AH101
- Dell Visor
- Asus HC102
- Acer OJO 500
Hololens Unity Setup
Available tools and SDKs
Tool/SDK | Description |
---|---|
OpenXR Developer Tools | Provides a demo scene that exercises various features of OpenXR, along with a System Status page that provides key information about the active runtime and the current headset. |
OpenXR Spec | Describes what OpenXR is, what capabilities and features it offers, and how to implement it into your own projects. |
OpenXR Loader | Discovers the active OpenXR runtime on the device and provides access to the core functions and extension functions that it implements. |
Examples
Hololens Unity Development
Feel free to play around with the sample app to get a feel for what's possible with Native development and Mixed Reality.
- BasicXrApp - Demonstrates a simple OpenXR sample with two Visual Studio project files, one for both a Win32 desktop app and one for a UWP HoloLens 2 app.
Deploy From Unity To Hololens
You can also watch a 60-minute walkthrough of BasicXrApp that tours all key components of the OpenXR API in Visual Studio:
What's next?
Now that you have a technology path in mind, let's equip you with the tools you need to start your journey.