Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Using Windows 10 Face detection for Hololens application

Would it be possible to use the windows 10 face detection and camera API to detect faces along with being able to place the face rectangles generated onto people in a Unity 3D application for hololens? Any ideas as to how this would work or what I would need to do to make this functional?

Best Answers

Answers

  • Does the Hololens even have a camera preview to apply effects to such as a face rectangle?

  • FrebinFrebin
    edited January 2017

    @neonninja

    Good attempt.

    The marker wont show in the correct position if there are multiple faces detected in different locations.

  • Hi @TimK,

    I tried building the sample code in holographic face tracking and got errors as follows:

    1) constexpr function return is non-constant error in functions in DirectXMath.h:

    inline XM_CONSTEXPR float XMConvertToRadians(float fDegrees) { return fDegrees * (XM_PI / 180.0f); }
    inline XM_CONSTEXPR float XMConvertToDegrees(float fRadians) { return fRadians * (180.0f / XM_PI); }

    2) static assertion failed with "Model constant buffer size must be..." in ShaderStructures:

    static_assert((sizeof(CubeModelConstantBuffer) % (sizeof(float) * 4)) == 0, "Model constant buffer size must be 16-byte aligned (16 bytes is the length of four floats).");

    static_assert((sizeof(QuadModelConstantBuffer) % (sizeof(float) * 4)) == 0, "Model constant buffer size must be 16-byte aligned (16 bytes is the length of four floats).");

    I am new to microsoft development and would like to ask some help in resolving these errors.

    Thank you and I am hoping for your kind response. :)

    Regards,
    Trish

    @TimK said:
    All Windows 10 Universal APIs, including face detection/tracking, can be used with HoloLens. The Windows.Media.Capture APIs allows streaming of preview frames from which face detection can be performed.

    I recommend starting with the BasicFaceTracking sample located here: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/BasicFaceTracking

    The sample only builds a 2D app, but it demonstrates how to stream preview frames from MediaCapture and run the FaceTracking on them. Please note there are two different "face detector" classes:

    • Windows.Media.FaceAnalysis.FaceDetector is intended for finding faces in a single image or frame
    • Windows.Media.FaceAnalysis.FaceTracker is optimized to track faces in a video stream, which is what you want to use in this case

    However, to use face detection in 3D space is a little tricky since all you'll get from these APIs is a 2D bounding box in pixel space. One possibility is to simply estimate the distance of the face based on the size of the bounding box in the frame image. If you're simply "tagging" faces with a simple graphic or lable, this might be good enough.

    Another option is to use the Sharing experience from Holograms 240 class. You can map the faces returned by FaceTracker with the 3D positions of participants in the session to render some effect on or around their faces.

    Be aware the output of FaceTracker is pretty noisy, and the size of the bounding box will quickly change depending on the orientation of the person's face, e.g. a profile view of the face will give you a tall, slender box while a frontal view produces a more square box. You'll probably need to perform some filtering of the face tracking output in your app.

  • YixunYixun ✭✭

    I can build, deploy and run this project on Emulator. But get error when I deploy it to HoloLens.
    The error is:

    DEP0700 : Registration of the app failed. Deployment Register operation with target volume C: on Package Microsoft.SDKSamples.HolographicFaceTracker.CPP_1.0.0.0_x86__8wekyb3d8bbwe from: (AppxManifest.xml) failed with error 0x80073CFD. See http://go.microsoft.com/fwlink/?LinkId=235160 for help diagnosing app deployment issues. (0x80073cfd)

    Can anybody help me to fix this?

    Thanks a lot.

    YL

Sign In or Register to comment.