HoloLens & OpenCV

Hello Everyone!

I'm not sure if this is the right place for this. If not feel free to delete it. Also if you know any better places to talk about this question I would like to hear them.

I am currently preparing a project at university and we thought about using OpenCV for the HoloLens to detect objects. There is this NuGet that provides OpenCV DLLs that should work with the HoloLens https://www.nuget.org/packages/OpenCV-Hololens/ and here is a tutorial how to communicate between Unity C# Scripts and a c++ DLL that works with opencv https://thomasmountainborn.com/2017/03/05/unity-and-opencv-part-three-passing-detection-data-to-unity/

So I used these to build the HoloLens DLLs and created my own DLL that I included into Unity. The Communication works without any problem. While testing it throws an AccessViolationException when I tried to get a Frame from the VideoCapture, so I tried to get some other info from the OpenCV VideoCapture but it returned 0x0 for the resolution - so it seems that doesn't work.

Then I changed my program to get a picture in Unity ( https://docs.microsoft.com/en-us/windows/mixed-reality/locatable-camera-in-unity ) and sent that to openCV and now it shows that the DLL throws an Exception when putting the Color32 Array into a Mat. I tried the same without the HoloLens and normal OpenCV actually is very forgiving when creating a new Mat with an Array.

Does anyone have any clue what I'm doing wrong? Maybe I am missing something trivial. The DLLs in the NuGet seem a bit old. I found some other sources that this task of making OpenCV work on the HoloLens is pretty much solved, but I'm running against walls here and slowly I'm getting out of ideas.

If anyone has any Input, let me know!

Thanks in Advance!


  • Does your app have the uwp app right to access the camera?

  • Jimbohalo10Jimbohalo10 ✭✭✭
    edited October 2018

    There is numerous entry for Open CV on HoloLens
    Basically most of the code is spread around GitHub and pointers are littered around the forum.
    Basically take the GitHub Repo rebuild it with Visual Studio Community then you can what you want. Your best place to get this is from the Spectator for HoloLens project see Spectator

Sign In or Register to comment.