Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Object recognition in Unity

Hi all,

The situation of my project:

  • I need to recognize an object(almost like a cube) in the real world with size 75cm x 50cm x 50cm.
  • The object is mounted on a wall.
  • I don't want to use any markers.
  • I have a 3D model of the object available.

As far as I see, there are two possibilities to reach this:

  • The object is almost completely white, so Vuforia is something that probably will not work.
  • Compare the spatial mapping with the 3D model.

I am going to try the second approach, something like this:

Since the object is 50cm wide, it should be possible.

Another video of similar functionality is made by Bentley:

If you haven't seen this video, watch it!
In the comments they state

Yes, the pipe/valve model is aligned first with the physical world version, before the maintenance can start

This is basically the approach I want to take too.

I am using Unity and the HoloToolkit.
https://developer.microsoft.com/en-us/windows/holographic/spatial_mapping_in_unity#shape_queries

Note that the shape analysis works on horizontal surfaces only. A couch, for example, is defined by the flat seat surface and the flat top of the couch back.

This implies that recognizing a cube is not possible using the standard API.

Any ideas on how to do this?

Best Answers

Answers

  • Vuforia is actually really decent at object recognition, though you're right that if it is pure white or has no visible markings it may be very difficult to recognize. That's likely going to be a challenge no matter what tech you go with. But 50 cm is pretty decent size. I would at least give Vuforia object recognition a shot and see what happens.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • Jimbohalo10Jimbohalo10 ✭✭✭
    edited December 2016

    I tried a white sheet of printer paper as a detection mask it works fine.
    My only problem came when there were multiple white pieces of paper in the detection window. I got moving tracking hits.

    HoloLens is supported by Vuforia by just changing the ARCamera prefab from Webcam to Holographic. Its important to have at least
    vuforia-samples-eyewear-unity-6-1-17.zip v6.1.17 to Support HoloLens

    The picture seen in the video is show in a YouTube video describing how to take a still picture and use this as a trigger to Vuforia to call an event on detection the disassembly blow up of the pump to be fixed is shown.

  • @jbienzms said:
    Vuforia is actually really decent at object recognition, though you're right that if it is pure white or has no visible markings it may be very difficult to recognize. That's likely going to be a challenge no matter what tech you go with. But 50 cm is pretty decent size. I would at least give Vuforia object recognition a shot and see what happens.

    If I choose to use the 3D mesh representation approach, color doesn't matter.
    I tried to use Vuforia as a test, with a sample A4 marker, but thought that recognizing a real object will be much harder.
    Do I just use an image of the object as image target?
    The Vuforia docs state that if you use a non A4 image target, append white. This would be impossible for me since the object itself is also white.

    @Jimbohalo10 said:
    The picture seen in the video is show in a YouTube video describing how to take a still picture and use this as a trigger to Vuforia to call an event on detection the disassembly blow up of the pump to be fixed is shown.

    To which picture are you referring?

  • If I am reading the comments in that Bentley video correctly they may not be using markers but it also seems as though they are not using object recognition. This sounds as though they are just making the user drag the 3D model of the pipes and align them visually over the real pipes before they do the checklist. I think I'd rather use markers in that case like the Lowes example.

    Taqtile

  • @mark_grossnickle said:
    If I am reading the comments in that Bentley video correctly they may not be using markers but it also seems as though they are not using object recognition. This sounds as though they are just making the user drag the 3D model of the pipes and align them visually over the real pipes before they do the checklist. I think I'd rather use markers in that case like the Lowes example.

    After reading it a second time I thought the same, although they don't seem to mention it explicitly.
    Markers are basically my Plan B/C if others really don't work.

    Shape analysis shouldn't be too hard, right? The thing is I don't know how to start.

  • @jbienzms said:
    For most of the demos I've seen, a VuMark or Image Target has been sufficient for decent alignment. We did a demo of aligning several holograms on a large 5 foot by 5 foot map and we just used a 5" image target for alignment. It wasn't perfect and was a little jittery during the alignment phase, but when we got the alignment good enough we had a voice command that disabled the ImageTarget behavior. This leaves the objects in Unity space and lets the HoloLens tracking take over. Since our map never moved, this was sufficient for us and it looked great.

    I am finally able to test the image target approach, and I have to say it works quite good!
    I made sure I got recognizable features and made a picture, cropped it and it worked.
    However, the visualization is a bit in front of the actual object, but I think I have to tweak some things.
    For a quick test, it works surprisingly well.

    Thanks!

Sign In or Register to comment.