Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Annotate to HoloLens HUD via UWP Desktop Application?

Hello Everyone,

I am trying to find a solution for the project I am building in Unity. I have a particular task that is giving me some trouble that requires sharing HoloLens data with a PC.

In a two-user scenario, I would like to have my Surface Pro 4 user be able to draw on my HoloLens user's HUD. This would be similar to what is demonstrated in Skype, but I want to make sure the annotations are projected directly on their colliding surface. Generating a gameObject on a particular surface is no big issue, but knowing where the surface is requires HoloLens to send the spatial mapping data out to the tablet user.

My question is this:

How do I send a combined live camera feed and live spatial map from my HoloLens to my Surface Pro 4 from within the multi-user Unity application?

Note: I will not be using the Surface as an AR device. Only for placing objects in the HoloLens user's view.

Note: It is not necessary that I be able to draw, just able to place icons. (I will likely have a library of icons that the Surface user can place, but the icons need to attach themselves to a colliding spatial mapped surface in order to be useful. I can't have them floating in space.)


  • Hi @Ereptor

    You can accomplish most if not all of what you're looking to do with the Remoting Host sample. In this sample the HL sends spatial mapping to an application, and your application creates the UI to send back to the HL. It does this over WiFi only. You can also determine what a user is looking at through gaze, and collisions. Think of it as an extended Holographic experience.

    Basically you run the Holographic Remote app on the HoloLens, and then your application connects to the HoloLens, and your application generates the experience. The rendering and power come from your desktop where the application runs, the spatial mapping, gaze and gestures come from the HoloLens. The HL basically becomes a extended mouse + monitor to your application.

    The Live camera feed may be the issue you run into, but only because you have to figure out how to do it. I know it's possible because you can run the remoting app + the Live capture at the same time.

    Dwight Goins
    CAO & Founder| Independent Architect | Trainer and Consultant | Sr. Enterprise Architect
    MVP | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer

  • Thank you, Dwight. As a Unity developer, I am grossly unfamiliar with development tools outside of the Unity editor and typical monobehavior framework of Unity's scripting assets. Scripts that can be integrated into an existing Unity project, or packages which can be imported as assets to Unity's project hierarchy would be super helpful.

    We do have some folks on the team who are more familiar with non-Unity work, but we have always been on such disparate projects that our skills rarely come in contact with one another. I'll pass these resources on to them to see if they can help, but the timeline on my current project is such that I doubt it will all come together in time.

    That being said, this is certainly helpful. I'll see whether I can integrate these pieces with my Unity scripts without slowing down my own development. I'm slowly and reluctantly coming over to VisualStudio from MonoDevelop because we are targeting HoloLens, but it only makes me ever more aware of how much I don't know about development outside of my Unity bubble.

  • Exactly same problem I am also having. Kindly share If you get a solutions.

Sign In or Register to comment.