Where are the Hololens Stereo Matices in Unity?

terbo · January 2017

Is there any way in Unity to get the stereo view and projection matrices for the Hololens? I really want to save the previous frame's view and proj matrices for various effects. I've tried everything I can think of:
1) Camera.GetStereoViewMatrix and Camera.GetStereoProjectionMatrix are always identity.
2) The camera's transform/Camera.worldToCameraMatrix and GL.GetGPUProjectionMatrix do not produce correct View-projection matrices. In fact they are always the same for both cameras.
3) I've tried Material.GetMatrix("UNITY_MATRIX_V") and Material.GetMatrix("UNITY_MATRIX_P"), which always fail, likely because they are packed in Constant Buffers (UnityShaderVariables.cginc).
4) I've tried writing a native UWP dll that manually extracts camera poses from a HolographicFrame, yet HolographicSpace::CreateForCoreWindow barfs (SEH) because Unity has already done it for the app's CoreWindow.

Am I missing something? These seems like fundamental data that should be readily available.

terbo · February 2017

Uhg, neither of those comments are helpful. After some experimentation, the answer is,

For Object Drawing:

camera.GetStereoViewMatrix(Camera.StereoscopicEye) == UNITY_MATRIX_V.
GL.GetGPUProjectionMatrix(camera.GetStereoViewMatrix(Camera.StereoscopicEye) ) == UNITY_MATRIX_P.

Note: UNITY_MATRIX_V and UNITY_MATRIX_P will be set to the proper eye depending on the camera pass.

For Screen-space Drawing:

camera.GetStereoViewMatrix(Camera.StereoscopicEye) == unity_CameraProjection.

Note: UNITY_MATRIX_P will be useless because Unity is blitting with a screen-space quad.

Dwight_Goins_EE_MVP · January 2017

I can't say for Unity, however it is fundamental data that is readily available for DirectX C# or C++ app...

With that being said, there must be a way to get to it in Unity, what is your custom shader logic look like? From my understanding these UNITY_MATRIX_XX are shader variables and pretty much are available in custom shaders.

AmerAmer · January 2017

Unity has a built in shader function that handles the two matrices. I think its called SetStereo or something similar, where it initializes the view matrix based on the instance index. As for C++, LIke Dwight said, essentially instance id is used to pull transforms for either a 0 or 1 index from the eye matrices.

terbo · February 2017