Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.
Options

How to implement custom in-app spatial mapping?

shinyeyesshinyeyes ✭
edited May 2017 in Questions And Answers

Problem

I want my app to be isolated from the system-global space data. It means when I start my app, I don't want to load existing data. I don't want my scanned data to be added to the system-global space data.

I tried...

When I call SpatialMappingSource.Cleanup(), it will destroy all the meshes but it will reload them back from the system. Also the scanned data will be added to the system-global space data.

I noticed that SpatialMappingObserver is loading data from the system but I couldn't figure out how to stop that.

Thanks.
Br, Seyoung

Tagged:

Best Answer

Answers

  • Options
    thebanjomaticthebanjomatic ✭✭✭

    I don't believe there is a way to do this, the spatial mapping data is managed at the OS level and the SpatialMappingSource just provides a way to query and access the data that is managed by the OS.

    Can you elaborate on what your use-case is?

  • Options
    shinyeyesshinyeyes ✭

    Thank you @stepan_stulov and @thebanjomatic for your replies.
    For those who are looking into the same matter might find this post interesting.

  • Options
    Jarrod1937Jarrod1937 ✭✭✭
    edited May 2017

    You really need to explain what you're trying to do. The spatial mapping is merely giving you access to the scanned 3D data and represents it as a mesh. The end result is the spatial mapping is just that, a mesh. If you want to, you can add your own processing to this mesh, or even throw out the spatial mapping entirely and load your own mesh that you then want to use as a spatial mesh. What is your particular scenario? We can help you with that.

  • Options
    shinyeyesshinyeyes ✭
    edited May 2017

    Thank you @Jarrod1937 and I apologize for the late reply. For some reason I wasn't notified about your comment.

    I'm running a university research project.
    The idea is to access as much raw data as possible. However, this seems not to be possible using HoloLens because it will only give you processed mesh.
    But let's limit to the topic of this thread.

    Scenario: reconstruct 3D model of a building.
    Problem:

    1. SpatialMapping app(the opensource code given by MS) would crash after it loaded enough mesh(Usually about 7.0m x 7.0m)
    2. Even if it wouldn't crash, it won't load newer meshes anymore.

    My work-around:

    If I can't delete space data let's delete meshes that are loaded.

    1. Delete meshes, which are 50cm away from camera, after SendMeshes() is called.

    // SpatialMappingSource.cs
protected void CleanupAfterSend(bool destroyGameObjects = true, bool destroyMeshes = true)
{
    var handlers = SurfaceRemoved;
    var count = surfaceObjectsWriteable.Count;
    var i = 0;

    for (int index = 0; index < surfaceObjectsWriteable.Count; index++)
    {
        Helper.debug(System.String.Format("Cleaning {0}/{1} surface", index, count));
        SurfaceObject surface = surfaceObjectsWriteable[index];
        if (SpatialMappingManager.Instance.isSurfaceNearCamera(surface.Filter.sharedMesh.bounds))
        {
            continue;
        }
        if (handlers != null)
        {
            handlers(this, DataEventArgs.Create(surface));
        }
        CleanUpSurface(surface, destroyGameObjects, destroyMeshes);
        surfaceObjectsWriteable.RemoveAt(index);
        i++;
    }
    Debug.Log(System.String.Format("CLEANED {0} surfaces out of {1}", i, count));
}

    // SpatialMappingManager.cs
public bool isSurfaceNearCamera(Bounds bounds, float radius = 0.5F)
{
    //var distance = Vector3.Distance(bounds.center, Camera.main.transform.position);
    var heading = bounds.center - Camera.main.transform.position;
    var distance = heading.magnitude;
    return distance < radius;
}

    Result

    Never reach the full cycle of a for-loop.

    It kinda works.

    Helper.debug(System.String.Format("Cleaning {0}/{1} surface", index, count));

    This debugging line never reached n/n surface. It would almost always stop at (n/2)/n surface.

    2. I also noticed that after removing meshes, the surfaceObserver will trigger callback method SurfaceObserver_OnSurfaceChanged which reloads the deleted meshes.

    So I add surfaces to surfaceWorkQueue that are near camera in order to avoid reload too much unnecessary meshes.

    private void SurfaceObserver_OnSurfaceChanged(SurfaceId id, SurfaceChange changeType, Bounds bounds, DateTime updateTime)
{
    switch (changeType)
    {
        ...
        case SurfaceChange.Updated:
            if (! isSurfaceObsolete(id) || SpatialMappingManager.Instance.isSurfaceNearCamera(bounds))
            {
                surfaceWorkQueue.Enqueue(id);
            }
            break;
        ...
}

    Mesh vs. Camera distance is not accurate.

    isSurfaceNearCamera() might say that a mesh which is 7 meters away from you is within 50cm. So it sometimes reloaded meshes that are far.

    Summary

    Is my method reasonable reconstructing indoor space of a building? In my solution, there's a problem that the surface data is still loaded in RAM even meshes are deleted.

  • Options
    Jarrod1937Jarrod1937 ✭✭✭

    Alright, so you're wanting to use the Hololens as a 3D scanner of sorts, correct? My experience is purely within Unity and the Hololens, but I can certainly tell you I have scanned areas larger than 7 x 7 m, with no crashes. From my understanding, once it runs out of space (storage I assume) it merely drops the old and saves the new (sort of like a FIFO buffer) Even then, you should able to to grab the meshes outputted while the scanning occurs and save them (whatever format you wish) along with positional data to place them n the correct spot. You can then reload these back if you wish. I'd imagine the raw data probably wouldn't be too useful anyways, a processed mesh with a list of verts in a serialized format should be the ideal output, given your requirements.

Sign In or Register to comment.