Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Locatable camera bugs?

I'm attempting to draw a ray from the camera's perspective in world space, but something is wrong with the output. Basically if I aim the centre of the camera image at a point in the world then draw a ray based on the output, it's not quite in the right place. I figured it might be related to intrinsic parameters such as centre of projection, so I grabbed the matrix with PhotoCaptureFrame.TryGetProjectionMatrix and I got the following:

******Instric Params Matrix*******
1.5182 0.0000 0.0094 0.0000
0.0000 2.6957 -0.0256 0.0000
0.0000 0.0000 -1.0000 0.0000
0.0000 0.0000 -1.0000 0.0000

However, the Locatable camera article ( https://developer.microsoft.com/en-us/windows/holographic/locatable_camera ) says this is the format:
Matrix4x4 format Terms
m11 m12 m13 m14 fx 0 0 0
m21 m22 m23 m24 skew fy 0 0
m31 m32 m33 m34 cx cy A -1
m41 m42 m43 m44 0 0 B 0

which has a few obvious inconsistencies, such as -1 as a constant in the format whereas the output I got is 0.

To draw the ray using the camera's image I'm doing the following:

photoCaptureFrame.TryGetCameraToWorldMatrix (out extMatrix)
mWorldPos = extMatrix.MultiplyPoint(Vector3.zero);
mWorldRot = extMatrix.MultiplyVector(Vector3.back);
...
GL.Color( Color.green );
GL.Vertex3 (mWorldPos.x, mWorldPos.y, mWorldPos.z);
Vector3 camEndPos = mWorldPos + (mWorldRot*5.0f);
GL.Vertex3 (camEndPos.x, camEndPos.y, camEndPos.z);

which is getting close, but definitely not as close as it should be. Is there an error here? The only thing I could think of to correct the error was the intrinsic parameters, but they're inconsistent with the documentation so I'm not sure if they're providing anything useful.

Answers

  • Hi everyone, hello Microsoft, please check this post:
    as Elspin says, in your documentation of the Locatable camera https://developer.microsoft.com/en-us/windows/holographic/locatable_camera, there is this:

    Matrix4x4 format Terms
    m11 m12 m13 m14 fx 0 0 0
    m21 m22 m23 m24 skew fy 0 0
    m31 m32 m33 m34 cx cy A -1
    m41 m42 m43 m44 0 0 B 0

    indeed I think this is wrong, it should be transposed:

    Matrix4x4 format Terms
    m11 m12 m13 m14 fx skew cx 0
    m21 m22 m23 m24 0 fy cy 0
    m31 m32 m33 m34 0 0 A B
    m41 m42 m43 m44 0 0 -1 0

    would anyone confirm?

    What is A?
    In my experience, and according to literature, the element 3,3 of projection matrix is 1. Not -1; is it related to a change in right/left cartesian coordinates? maybe from camera (right handed with z going backward) to unity (left handed with z going forward)? support is appreciated

    b.r.
    Gio

  • Thanks for the tip @gio , that looks like it might make sense. Hopefully someone from Microsoft looks at this. There badly needs to be a way to report bugs to them other than them maybe one day looking at the forums

  • According to the documentation of the locatable camera, I also would like to know what is the meaning of the Parameters A and B in the projection Matrix?

Sign In or Register to comment.