The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
How to Select a Real World Object
Hi,
HoloLens API currently provides an easy way to "select" a hologram, through gaze and gesture/voice, for instance.
Is there a way to select a real world object, in a similar way? Clearly, this will in general require some type of object detection, if not recognition. But, I want something "quick and dirty" so that I don't have to spend months coding. In fact, I am really interested in "selecting a (small) region" from a scene (e.g., a still image) at the moment. I am thinking of something like a "rubber band" UI in GUI systems. (That is, we can let the user select a (2-D projected) region that contains an object of interest instead of selecting the object itself. Just an idea.)
Has anybody tried something like this with HoloLens? Any advice?
Thanks,
~H
Best Answers
-
Options
Patrick
mod
If you can detect the object, then making it selectable is the easy part. The easiest way of doing real world object detection right now is probably QR Codes + OpenCV.
===
This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.(Daddy, what does 'now formatting drive C:' mean?)
5 -
Options
With the room sensor you can get a mesh, which is a start. You'd want to post-process that mesh in some way (a non-trivial task), create colliders for the post-processed bits, and then you can interact with them. While Unity does have a Mesh collider, with the limitation of being convex for many interactions, I've found it to be less than ideal for some of these more real-world scenarios (There are Unity tools in the store which will take a solid and make a series of colliders so it behaves in a concave way).
5
Answers
If you can detect the object, then making it selectable is the easy part. The easiest way of doing real world object detection right now is probably QR Codes + OpenCV.
===
This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.
(Daddy, what does 'now formatting drive C:' mean?)
With the room sensor you can get a mesh, which is a start. You'd want to post-process that mesh in some way (a non-trivial task), create colliders for the post-processed bits, and then you can interact with them. While Unity does have a Mesh collider, with the limitation of being convex for many interactions, I've found it to be less than ideal for some of these more real-world scenarios (There are Unity tools in the store which will take a solid and make a series of colliders so it behaves in a concave way).
Thanks for the input. @Patrick I'm looking into visual fiducial systems like AprilTags. @dracolytch That's an interesting idea. It may be somewhat limiting for our use cases (e.g., we cannot detect "2d objects" like a car drawn in a picture, or even a business card placed on a desk, this way), but I think still it can be rather useful. (Obviously, the question is whether the spatial mapping resolution is fine enough for object detection purposes. What's the minimum size of features that can be detected through spatial mapping data? etc.) Thanks. ~h
The minimum size appears to actually depend on how close you get to the object... So the correct, but not-fun answer is "It depends". I've pulled the mesh from my current office, and I've been able to clearly identify mugs, and other objects of similar size.
That's good to know, @dracolytch. I'll play around with my device, and see what I can do with spatial mapping. Thanks for the suggestion, ~h
I can identify the objects. What are the steps for spatial mapping?