Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Can Hololens's Locatable camera be used for more than one process?

Hi! I am trying to use Vuforia Hololens support while using the camera's byte stream data to do some image processing, but the program exit with error code 255 when I tried to start the PhotoCapture process. Earlier today when I tried to run the Vufoira and live preview at the same time, the program also exit with error code 255. Since Vuforia need the camera data for object recognition, so I am wondering if the camera data can only be used at one place? Thanks a lot!

Answers

  • ContextVRContextVR ✭✭✭

    @Zen This would be a good question for the folks at PCT, as they are always looking for feedback from HoloLens EAP developers, so give them a shout.

  • @ContextVR Thanks for your reply! So how should I contact them? Sorry I am new to the forum haha.

  • ContextVRContextVR ✭✭✭

    If you a part of the EAP, you probably have a contact who gave you access. They recently reached out asking for feedback, I would just respond to that email.

  • @Zen I am also interested in camera related App. But what I am doing now is to take a picture and then process them. I did not find a way to process video string data. So when I saw you said "camera's byte stream data", I am wondering what you mean? Do you mean that you have access to every frame of the camera data?

  • @Fireman Yes, since I am trying to do image processing for the image for every frame. Now the problem I have right now is whenever vuforia is at play I can't start photo taking mode, and though I can get the camera frame from a Vufoira callback I can't access the metadata from photocaptuerframe that contains the toWorldMatrix. I tried to calculate the toworldmatrix by calculating the offset of the Unity world main camera and the webcam but I have little luck. Any thought?

  • @ContextVR Thank man!

  • @Zen could you share how to access each frame? Do I have to use Vufoira to do that?

  • @Fireman So if not using Vuforia we can just call capturephoto every update call. With Vuforia since the camera is occupied(Or I think that is the reason) I can't even start the photo capture mode. So I have to use one of Vuforia's callback function to access each frame.

  • ContextVRContextVR ✭✭✭

    @Zen You could also try OpenCV image recognition and pose estimation, but Vuforia is definitely easier. Have you also tried AR Toolkit by chance?

  • @Zen You can use a WebcamTexture of Unity3D to open HoloLens locatable camera, call Texture2D.GetPixels32 to get the pixel array of current frame, and then do whatever processing on it.
    But keep the image processing process in async mode, in order not to block the Unity UI thread.
    I am using this technique for integrating ARToolKit wit HoloLens.
    This is my repository: https://github.com/qian256/HoloLensARToolKit

  • We made an open source project called CameraStream to make it easier to get the HoloLens locatable camera as a video stream in Unity. You can get access to the byte array in real time and do whatever you'd like with it.

    The class was created to look similar to Unity's VideoCapture class so that developers understand easily how to implement it. Hopefully this helps some people!

Sign In or Register to comment.