Obtain a specific Mesh

Hello everyone,

I am developing a game that is played on a table. We have a script that scans dynamically in order to detect objects that are been placed on it (real objects). However, we don't know how to obtain the mesh of these objects. How could I do it?

Thanks in advance,
Cristian.

Best Answer

Answers

  • Hey, Christian.

    This task is hard. HoloLens operates surfaces, not objects. So to extract an object from a continuous, distorted, derivative and pretty low-poly surface (some objects are seen as pretty much a pile of like 10 triangles) one requires a fair deal of (spatial) mesh processing. And even that only for some special cases like something overall convex on something overall flat.

    As far as I know spatial processing in the HoloToolkit can detect vertical and horizontal surfaces so you could try to subtract them from the whole surface and see what's left.

    Perhaps you don't need to detect the objects and can "cheat" in another way to achieve what you require? For instance you could try to detect a concentration of triangles in your spatial mesh that are all grouped together and that all have their normal vector non-vertical?

    It is no wonder that the vast majority of HoloLens apps simply offload the task onto the human to point onto the floor or wall.

    What is the problem you're trying to solve?

    Mixed reality developer based in Munich. Building a holographic navigation system within the paradigm of industry 4.0 and trying to be helpful on the forums. Write me if you have an idea in mind to brainstorm or collaborate on. Cheers

  • edited November 15

    Hi @stepan_stulov ,

    We are developing an app that is played over a table as I said before. The application starts scanning the room in order to detect a table. Once the scan time has finalished, the table plane is created. Afterwards, we begin to scan dynamically to do the process that I commented previously.

    We have a box collider on the table that simulates the region where we want to detect objects. We check if the set of vectors (mini cubes) provided by the spatial mapping mesh filters is inside it. The problem is that, in unity, the vectors appear correctly drawed. However, when we test it in the HoloLens, we don't detect any changes inside the box.

    In the following photo we can see the table, the box and the mini cubes

    The following code shows how we gather the meshes to process them.

    List meshFilters = SpatialMappingManager.Instance.GetMeshFilters();

        for (int i = 0; i < meshFilters.Count; i++)
        {
            Vector3[] vertices = meshFilters[i].mesh.vertices;
            for (int j = 0; j < vertices.Length; j++)
            {
                Vector3 posVertice = vertices[j];
             // if the posVertice is inside the box do whatever
            }
         }
    

    Thanks in advance,
    Cristian.

  • stepan_stulovstepan_stulov ✭✭✭
    edited November 15

    It may be that you're not detecting the changes because you're still referencing an old mesh while spatial mapping has been fed with a new one? What's the strategy for fetching the mesh you're using? Update()? Start()?

    Additionally there is a flag on SpatialMappingCollider something like FreezeUpdates. Make sure updates are not frozen.

    Mixed reality developer based in Munich. Building a holographic navigation system within the paradigm of industry 4.0 and trying to be helpful on the forums. Write me if you have an idea in mind to brainstorm or collaborate on. Cheers

  • Hi @stepan_stulov ,

    We have focused the problem from other point of view. Instead of obtainig all meshes, we have used the 'OnTriggerStay' method to get the collider that is inside the box. However, we have noticed that the collider, which 'OnTriggerStay' detects, is the same as the collider of the mesh which the process of mapping generated. Therefore, when we want to obtain the size of this collider, it gives us the size of the full mesh. How can we get the size of the collider, which is inside the box, taking this problem into account?

    Thanks a lot for your help!
    Cristian.

Sign In or Register to comment.