The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Get position & rotation of VideoMediaFrame
holoben
✭
This website and this sample code show how to access the video frames using the latest Windows 10 release, version 1607 (14393 build). I'm considering to do it this way because Unity currently does not provide a way to access the video camera at 30fps (except for saving the video to disk, which I do not want to do). However, once I get the video frame VideoMediaFrame, I'm not sure how to get the position and rotation of that video frame. I know I can access VideoMediaFrame.FrameReference.CoordinateSystem, which returns a SpatialCoordinateSystem, but even if I can get that, how do I get the position & rotation from that?
I'm assuming that if I use DirectX, there's a way to use that SpatialCoordinateSystem for getting the frame's position & rotation. But if I'm using Unity, is there a way to do this?
Answers
In DirectX, yes you typically have the WorldViewProjection matrix, and your keeping track of it during each frame, because you're going to transform it based on the spatialCoordinateSystem vertexpositionscale you get from the HL. You can do in in code or in the HL shader. You get the position from the transposed normal world matrix.
All this to say... I guess in Unity you have to do the same. I don't know how you access it, maybe look for the camera object (as that's usually the worldViewProjection), try to see if you can get a matrix from that to do the math, usually Unity has a function or mechanism to do this, check this post out.
Dwight Goins
CAO & Founder| Independent Architect | Trainer and Consultant | Sr. Enterprise Architect
MVP | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer
http://dgoins.wordpress.com
Thanks @Dwight_Goins_EE_MVP. I'm hoping Unity has a way to do this, or at least someone would know if Unity plans to implement a way to get 30fps RGB video footage (not saved to disk) with access to the camera pose for each frame. I don't want to migrate to DirectX if I don't have to...