Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Locatable camera bugs?

I'm attempting to draw a ray from the camera's perspective in world space, but something is wrong with the output. Basically if I aim the centre of the camera image at a point in the world then draw a ray based on the output, it's not quite in the right place. I figured it might be related to intrinsic parameters such as centre of projection, so I grabbed the matrix with PhotoCaptureFrame.TryGetProjectionMatrix and I got the following:

******Instric Params Matrix*******
1.5182 0.0000 0.0094 0.0000
0.0000 2.6957 -0.0256 0.0000
0.0000 0.0000 -1.0000 0.0000
0.0000 0.0000 -1.0000 0.0000

However, the Locatable camera article ( https://developer.microsoft.com/en-us/windows/holographic/locatable_camera ) says this is the format:
Matrix4x4 format Terms
m11 m12 m13 m14 fx 0 0 0
m21 m22 m23 m24 skew fy 0 0
m31 m32 m33 m34 cx cy A -1
m41 m42 m43 m44 0 0 B 0

which has a few obvious inconsistencies, such as -1 as a constant in the format whereas the output I got is 0.

To draw the ray using the camera's image I'm doing the following:

photoCaptureFrame.TryGetCameraToWorldMatrix (out extMatrix)
mWorldPos = extMatrix.MultiplyPoint(Vector3.zero);
mWorldRot = extMatrix.MultiplyVector(Vector3.back);
...
GL.Color( Color.green );
GL.Vertex3 (mWorldPos.x, mWorldPos.y, mWorldPos.z);
Vector3 camEndPos = mWorldPos + (mWorldRot*5.0f);
GL.Vertex3 (camEndPos.x, camEndPos.y, camEndPos.z);

which is getting close, but definitely not as close as it should be. Is there an error here? The only thing I could think of to correct the error was the intrinsic parameters, but they're inconsistent with the documentation so I'm not sure if they're providing anything useful.

Answers

  • Hi everyone, hello Microsoft, please check this post:
    as Elspin says, in your documentation of the Locatable camera https://developer.microsoft.com/en-us/windows/holographic/locatable_camera, there is this:

    Matrix4x4 format Terms
    m11 m12 m13 m14 fx 0 0 0
    m21 m22 m23 m24 skew fy 0 0
    m31 m32 m33 m34 cx cy A -1
    m41 m42 m43 m44 0 0 B 0

    indeed I think this is wrong, it should be transposed:

    Matrix4x4 format Terms
    m11 m12 m13 m14 fx skew cx 0
    m21 m22 m23 m24 0 fy cy 0
    m31 m32 m33 m34 0 0 A B
    m41 m42 m43 m44 0 0 -1 0

    would anyone confirm?

    What is A?
    In my experience, and according to literature, the element 3,3 of projection matrix is 1. Not -1; is it related to a change in right/left cartesian coordinates? maybe from camera (right handed with z going backward) to unity (left handed with z going forward)? support is appreciated

    b.r.
    Gio

  • Thanks for the tip @gio , that looks like it might make sense. Hopefully someone from Microsoft looks at this. There badly needs to be a way to report bugs to them other than them maybe one day looking at the forums

  • According to the documentation of the locatable camera, I also would like to know what is the meaning of the Parameters A and B in the projection Matrix?

Sign In or Register to comment.