Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

How to detect face with MS Cognitive Services Face API?

Hello All,

how to detect Live person face in single game object with cognitive service.

Thanks
Ankit Sangani
Skapy : ankit.sangani4

Answers

  • utekaiutekai ✭✭✭
    edited October 2016

    I'm working on this too as well with computer vision api. There are samples available.

    This one is a c++ sample, https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/HolographicFaceTracking . This does real time face detection. When none found displays a text object 'No faces detected' and when found shows a video frame of the face in a moving object that sticks with main camera.~~~~

    https://forums.hololens.com/discussion/1989/using-windows-10-face-detection-for-hololens-application

    This sample relies upon opencv, but won't run on hololens yet https://github.com/Microsoft/Cognitive-Samples-VideoFrameAnalysis/


    More direct to your question. One technique is to use the locatable camera to take pictures and send those to the face api.

    Here's instructions for getting pictures (though this snaps a photo at startup, it can be easily modified to be activated with voice to take a picture any time you desire).

    https://developer.microsoft.com/en-us/windows/holographic/locatable_camera_in_unity

    With the picture, you'd just call to the face api directly (perhaps using StartCoRoutine).


    If you want to see boxes around faces in realtime, it's a more extensive effort at this point.

  • Thanks @utekai for quick response.

    I am able to capture photo with (Locatable camera in Unity) as you mention.
    How do I integrate Or Call Face or Emotion API by sending photo into unity hololens c# application.

    Thanks,
    Ankit Sangani

  • utekaiutekai ✭✭✭

    @Ankit09_Sangani

    The initial approach I took was to use client library from Microsoft. Here's a C# review of calling to Face APIs, but the issue is the Microsoft.ProjectOxford libraries aren't built yet on .Net Core and won't run as UWP app and so won't yet run on hololens.

    These next two samples are already UWP apps though.

    Here's an example from the Windows 10 samples for basic face detection and includes c# implementation. This is a good sample to start with as the sample works off a static pic rather than video, though video can be used.

    Here's another Windows 10 sample for basic face tracking with c# which is working with a video stream.

    Either of these samples can be deployed to and run from a hololens as is, but aren't really meant to be used from a hololens, they're 2d apps.

    In the Basic Face Detection sample, the pertinent code that calls to the Windows.Media.FaceAnalysis.FaceDetector class is this:

    FaceDetector detector = await FaceDetector.CreateAsync();
    faces = await detector.DetectFacesAsync(detectorInput);

    Where detectorInput is a SoftwareBitmap object, or just an uncompressed bitmap.

    Here's the MSDN docs on the FaceDetector.

    So it's rather simple to this point. The next step would be to draw a visualization of the found faces in the picture sent, if any found.

  • I've built something like this and have made it open source at https://github.com/UoA-eResearch/hololens_facial_recognition

    When you air tap (or click with the clicker) it takes a photo and posts the image data to the Microsoft Face API, then displays the response as floating text next to any of the faces detected. Looks like this -

    Enjoy!

  • I am using this api in my javascript code. I am able to fetch the results for any image. But the api is taking 3-6 seconds to fetch the data. Is anyone getting a better response time?

  • utekaiutekai ✭✭✭

    I'm doing the computer vision api and it takes about that long for the first request then goes faster, maybe down to 1-2 seconds.

Sign In or Register to comment.