The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
how to detect a real object using hololens for unity?
I am trying to detect a real object and than place a 3D model of that object over it . Can anyone help me with how I can achieve this using unity
Best Answers
-
Optionsstepan_stulov ✭✭✭
Hey, @Uwp
To detect something in real world you can take one of these paths:
- Supplement the real object with a tracking marker and use one of the marker tracking frameworks, for example Vuforia.
- Ask the user to simply align the hologram with the real object using gaze and gestures.
- Recognize the real object from the sensory data input. For example by analyzing the spatial mapping mesh. Spatial mapping mesh is very coarse, so the more complex the real object's topology is, the harder this task. This only works well with simple planes, floors, walls. I don't recommend going this way unless you absolutely have to.
You can take one of these paths for initial localization of the hologram and later delegate the tracking to your HoloLens by attaching the WorldAnchor component to it. You can also persist this anchor between application relaunches (via WorldAnchorStore) or even transfer the anchors onto another HoloLens (via WorldAnchorTransferBatch).
Hope this helps.
Building the future of holographic navigation. We're hiring.
5 -
Optionsstepan_stulov ✭✭✭
It depends on what you call native. From within Unity not much is native anymore as those APIs are a bypass to UWP APIs. The closest I can think of is the SpatialUnderstanding in the MRTK framework. But it only finds horizontal/vertical surfaces. If you want to recognize something more complex then I believe using point cloud derivatives (spatial mapping) is a no go with HoloLens. You might want to go visual recognition direction. Perhaps this article can be helpful:
https://blogs.windows.com/buildingapps/2017/03/06/building-terminator-vision-hud-hololens/
Cheers
Building the future of holographic navigation. We're hiring.
5 -
Optionsstepan_stulov ✭✭✭
They don't use object recognition in that video. Check out the first question in the comment section and the video author's response to it. They pretty much take the (2) path from my original answer to you. They create a 1-to-1 copy of their pipe setup and simply align it with reality and then delegate further anchoring to HoloLens via a WorldAnchor. This is a very standard approach. Make no mistake, there is no object recognition in their approach. They use human brain and eyes to align.
I believe what you want is recognizing any ambiguous meshes and HoloLens aligning visual content against them with precision and performance. We all want that! But I'm afraid we (the industry) just aren't there yet. Check out this thread:
https://forums.hololens.com/discussion/1461/object-detection-hololens
Best quote from there: "HoloLens does not support any of detection, recognition, or measurement of objects. It would be the Holy Grail of Computer Vision: Detect, recognize, and measure objects."
Cheers
Building the future of holographic navigation. We're hiring.
5
Answers
Hey, @Uwp
To detect something in real world you can take one of these paths:
You can take one of these paths for initial localization of the hologram and later delegate the tracking to your HoloLens by attaching the WorldAnchor component to it. You can also persist this anchor between application relaunches (via WorldAnchorStore) or even transfer the anchors onto another HoloLens (via WorldAnchorTransferBatch).
Hope this helps.
Building the future of holographic navigation. We're hiring.
hi @stepan_stulov are there any native api which can be used for object detection in real time other than using external frameworks
@Uwp
It depends on what you call native. From within Unity not much is native anymore as those APIs are a bypass to UWP APIs. The closest I can think of is the SpatialUnderstanding in the MRTK framework. But it only finds horizontal/vertical surfaces. If you want to recognize something more complex then I believe using point cloud derivatives (spatial mapping) is a no go with HoloLens. You might want to go visual recognition direction. Perhaps this article can be helpful:
https://blogs.windows.com/buildingapps/2017/03/06/building-terminator-vision-hud-hololens/
Cheers
Building the future of holographic navigation. We're hiring.
@stepan_stulov hi thanks for the information,
I want to know is there any other frameworks like vuforia that supports object detection and uses unity and hololens. also how good is the vuforia in object tracking and is the alignment of a 3d model over the real object working accurately using vuforia.
i want to make somthing like this.
https://youtube.com/watch?v=QTuKcm8s4QQ
can you help me with how can i achieve this.
They don't use object recognition in that video. Check out the first question in the comment section and the video author's response to it. They pretty much take the (2) path from my original answer to you. They create a 1-to-1 copy of their pipe setup and simply align it with reality and then delegate further anchoring to HoloLens via a WorldAnchor. This is a very standard approach. Make no mistake, there is no object recognition in their approach. They use human brain and eyes to align.
I believe what you want is recognizing any ambiguous meshes and HoloLens aligning visual content against them with precision and performance. We all want that! But I'm afraid we (the industry) just aren't there yet. Check out this thread:
https://forums.hololens.com/discussion/1461/object-detection-hololens
Best quote from there: "HoloLens does not support any of detection, recognition, or measurement of objects. It would be the Holy Grail of Computer Vision: Detect, recognize, and measure objects."
Cheers
Building the future of holographic navigation. We're hiring.
@stepan_stulov thanks this information was really helpfull.
and can we align the 3d model over the real object exactly because the real object may not have a plan surface ?
Well, if you align using a human, you don't need any surfaces at all. Just attach the hologram to the camera and confirm by a gesture when the hologram visually aligns with the real object. WYSIWYG. Cheers
Building the future of holographic navigation. We're hiring.
@stepan_stulov Hi can we use point cloud library for object detection in hololens using unity?