Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Given a pixel, in an image taken by Hololens, how to find the 3D depth of the point in world space?

In the above link, we see how we can convert, a pixel on an image taken by camera to a 3d ray which determines the path taken by the photons that produced the pixel. How can we find the intersection of such a ray with the Spatial Mesh to find the corresponding 3D depth of the point? A solution on DirectX would be great. Can you also explain the variables in the above link i.e WorldSpaceRayPoint1 and WorldSpaceRayPoint2? I dont understand what they mean by "ray point in world space"


  • Hey
    I cant provide a solution in DirectX, but maybe some explanation helps your as well.
    Basically the "WorldSpaceRayPoint1" is your camera center. And the "WorldSpaceRayPoint2" is the pixel in world space. Therefore you can cast a ray from the camera center throught the "WorldSpaceRayPoint2". This ray will hit the real world a the position where the photons came from.
    Therefore you might find the real world 3D-position by performing a Raycast with the Ray resulting from the "WorldSpaceRayPoint1" and "WorldSpaceRayPoint2" and the spatial map. In Unity there is the possibility to use Physics.Raycase(...) to do this. I'm not sure about DirectX but I assume that there exists something similar.

Sign In or Register to comment.