Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Spatial mapping availability [updated support overview table]

Dear Mixed reality crew,

I am usually developing using Hololens at my company but first I was thrilled at the idea of having an immersive device for a decent price that "might" maybe have spatial mapping. From what I have heard however, the current devices seems to not support this. I can think of a lot of possibilities using a cheap device with spatial mapping even if it is not see-through and I think that could be a game-changer to Microsoft too!

Just to name a few uses:

  • Creating the spatial map by itself and a possibility to save it is worthy for a lot of people anyways.
  • If there is a spatial map, one can create virtual approximating environments of walls and surroundings. For example you can kind of recreate a version of Robo-Raid where you first scan your room and then a game would create textured sci-fi walls of a spaceship around you according to your scans or something like that.
  • For long range tracking - that is tracking beyond some meters where errors get too big - could be solved by accessing some kind of depth information.
  • etc.

I see that some people also wish to overlay virtual content of stereo camera view (which seems not possible neither), but I think the real key is not this, but the spatial mapping first. Having many immersive MR headsets also complicate this question even further.

I advise that we have a forum topic here where someone says in a table what each headset is capable of in this sense and whatnot. When things are not clear I am never buying devices on my own.

My current understanding is this (X: None, OK: Have, ?: no idea, please explain):
see_through spatial_map(C++) spatial_map(Unity) gestures head_tracking Hololens OK OK OK OK OK Acer X ? X X OK HP X ? ? X OK Lenovo ? ? ? ? OK Asus ? ? ? ? OK

If Microsoft people do not want to help us, at least we can update this forum by ourselves and share information in one place, but I hope they tell us and update us properly in this.

Thier Richard
AR/VR dev
Grepton Zrt.

Comments

  • Sorry, I see my carefully made (haha) initial table have collapsed so here is my understanding by adding a picture:

    Can you people update this picture accordingly???

    (at least the above original message of mine shows my initial confusion better)

  • Hello prenex,

    You have a really great table for this issue. However as far as I know, only HoloLens is supported for spatial mapping, here is the doc:
    https://developer.microsoft.com/en-us/windows/mixed-reality/spatial_mapping#device_support
    For the reason I suspect it is related to cost of the device.

    Best regards,
    Barry

  • james_ashleyjames_ashley ✭✭✭✭

    @prenex,

    Another way to look at it is that the MR headsets are inexpensive in part because they don't have spatial mapping\surface reconstruction capabilities. (I think we'll get there someday, though. Just not for a while.)

    James Ashley
    VS 2017 v5.3.3, Unity 2017.3.0f3, MRTK 2017.1.2, W10 17063
    Microsoft MVP, Freelance HoloLens/MR Developer
    www.imaginativeuniversal.com

  • Just to add a little more to why it's not likely that we will see spatial mapping in any of these early headsets, the spatial mapping/understanding processing all takes place on the HPU, or Holographic Processing Unit, on the HoloLens. This special chip takes all the input from the various sensors and cameras, and uses it to reason about the environment. There are no machines out there today that carry this special chip besides the HoloLens. This is why we are unlikely to see spatial mapping in the near future on headsets other than the HoloLens.

  • edited August 2017

    So I think this is totally possible...

    Why wouldn't the below idea be possible, with the right desktop hardware irregardless of which HMD device (HP/Acer/3Glasses)...

    You (MS basically) could access the Video camera, use stereo vision to determine depth, learn some tracking features based on depth, and color, implement a SLAM algorithm, and reconstruct your environment. If the device were connected to a powerful enough desktop system you could in theory make it real time, thus Spatial Mapping/ surface reconstruction...

    Maybe the only technical challenge is the throughput of data. Is a USB 3.0 protocol (capable of 5GB) big enough. If you have a learning algo that takes at least 5 different features, and if we want a accurate system we can easily reach the 2-3GB threshold of data coming in per second, which would basically kill frame rate if all the data is being used to do SLAM. - Just thinking out loud here.

    Dwight Goins
    CAO & Founder| Independent Architect | Trainer and Consultant | Sr. Enterprise Architect
    MVP | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer
    http://dgoins.wordpress.com

  • Note @fzambetta just posted on the Slack channel a link to a team who has done something similar with Stereo cameras: m/blog/index.php/2017/03/31/new-zed-sdk-2-0-introduces-spatial-mapping-and-more

    Dwight Goins
    CAO & Founder| Independent Architect | Trainer and Consultant | Sr. Enterprise Architect
    MVP | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer
    http://dgoins.wordpress.com

  • Imo, It is technically possible for the acer/hp headset to support a minimal spatial mapping (like floor only). Maybe future driver updates may realize this goal.

    Related to this thread, one of the MR app models supported by MS is called the blended app which would require spatial mapping ability in the immersive headset. In my view, the blended app is the true 'mixed' reality app as compared to the existing immersive app model.
    I am guessing the next gen of immersive headset would probably do this (plus it's still much cheaper than full HoloLens capabilities).

  • @prenex said:
    Sorry, I see my carefully made (haha) initial table have collapsed so here is my understanding by adding a picture:

    Can you people update this picture accordingly???

    (at least the above original message of mine shows my initial confusion better)

    I believe all the spatial mappings functionalities in DX native libs are available in the Unity MR sdk (?).

    Afaik, currently all immersive headsets do not support gestures and spatial mapping.

Sign In or Register to comment.