The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Active facial recognition on floating planes in 3D space
My dream Hololens app is an active facial recognition app that would display a floating plane next to a face (in its correct 3D space) with their name and some information. I would include photos of all of my students along with their information as the searchable faces and I would use it as a proof-of-concept in the classroom to be able to identify my students by name (I teach 150-300 students per semester) and likely produce an academic journal article on the experience.
I understand Microsoft has a facial recognition API available for developers:
https://www.microsoft.com/cognitive-services/en-us/face-api
Can Microsoft's API only recognize faces from still images or could it use the live video feed of the Hololens camera?
If something like this is possible, is this a project that would need to be created in Unity or in Visual Studio? Any advice from the community?
I have all of the tools and successfully tested the emulator environment yesterday. In Wave 2, so I'm not expecting hardware until June/July.
Thank you kindly for your help.
Answers
I believe it is possible. It will be very easiy to do as 2D UWP app but then, but IMO, you'll have a limitation of "floating pane" presentation surface which will be limited by the size of your virtual window. It is possible with Unity as well (using third party libraries supported) and with Unity you can place a "floating pane" anywhere in the 3D world.
@BluePotato This is very possible. I was actually in the process of working on this and ran across your blog post. Right now, Microsoft Cognitive services limits us to 30,000 captures a month; I am not sure on the video constraints, but it will also work with video. They provide an api to match two photos, and present back a percentage of likelihood that they are the same person. Using this API I'd like to assign names / notes about the user and display this information around them.
Glad to hear you're working on it! Best of luck to you.
I've built something like this and have made it open source at https://github.com/UoA-eResearch/hololens_facial_recognition
When you air tap (or click with the clicker) it takes a photo and posts the image data to the Microsoft Face API, then displays the response as floating text next to any of the faces detected. Looks like this -
Enjoy!
Hello , I've downloaded your program but it doesn't work. It doesn't ocurr anything when I run it on visual studio with the hololens.
Can you tell me how did you upload it to Unity, or maybe I didn't make something right. Thank you for your time.