Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

APIs for Inferring Physic Materials and Associated Accoustics/Physics of Spatial Mapped GameObjects

BotGreetBotGreet
edited May 2016 in Questions And Answers

So...thinking through the capabilities in evidence with my first week with the HoloLens, the lazy/time-limited developer in me has been brainstorming what APIs would be logically next or what third-party libraries to seek out that may just need adaptation to the HoloLens.

I see that a Custom Gesture Recognizer API doesn't exist yet for HoloLens from prior questions answered here, (nor the IK tools to develop these and train the system via machine learning to recognize custom gestures - assume that would be the approach due to the variability involved?)

In terms of adding to the wow factor, are there any examples of APIs or code that could be integrated to also infer and associate physic materials to Walls, Ceilings and Floors (may missed in the tutorials or elsewhere) with the HoloLens? E.g. A Photo capture sampling while creating the spatmap, and a machine learning algo can predict that the floor material fits the category of "berber carpet", "marble" or "ceramic tile" with typical coefficients of friction of X, and elasticity of Y, and should use or attenuate sound effect Z.

https://www.microsoft.com/cognitive-services/en-us/computer-vision-api

I have access to IBM and some other cognitive services, so hoping there is a way to constrain detection based on the typical uniformity of walls, floors and ceilings minus the objects...e.g. by chance sampling would be expected to pick those up more often in random sampling assuming you aren't in a hoarders' house...

Object recognition obviously will expand, but I think that is the starting place since there are limited possibilities for the broad categories of an indoor-space in X location. Speaking of which maybe there are some heuristics or probabilities that can be inferred as part of the probabilistic inference from geo-location data. E.g. in Arizona or Beijing, you are not likely to have carpeted flooring due to haboobs/dust levels. You're less likely to have a basement in Arizona versus Colorado...cross reference barometer/altimeter/GPS readings from a connected mobile app to infer what level of a single building you are in.

Or automated recognition of refrigerator / dish washer / stove top that the space should be inferred to be "kitchen" versus big large screen probably family room.

Thanks very much!

Tagged:
Sign In or Register to comment.