Observer memory growing indefinitely

Louis · July 2016

Hi Holofriends,

The application we are developing could take advantage of the continuous spatial mapping. I've seen it was possible in documentation, and it is actually working. Although, our application's memory is growing at a slow but steady rate, and eventually crashes due to out of memory exception. I'm pretty sure the issue is with the spatial mapping, as when I stop the observer, the memory stops increasing. I've read all the documentation about the spatial mapping, and followed the guidelines on how to make it work. Also, I'm using objects found in the holotoolkit as is (SpatialMappingManager, SpatialMappingObserver). I've done some investigation, and the issue seems to be with the surfaces not releasing memory they were allowed. I've made a function to test that :

public void ClearObserver()
{
    Cleanup();
    surfaces.Clear();
    surfaceWorkQueue.Clear();
    Resources.UnloadUnusedAssets();
}

Essentially, it destroys the GameObjects holding the surfaces, and removes references to them. Also, I tried to put the UnloadUnusedAssets to make sure it isn't Unity that is still holding the memory, but that didn't change a thing. Every time the function is called, unity says it is clearing X unused assets, but no memory is released. I've also tried to reproduce the problem in the Holograms 230: Spatial mapping tutorial (), by removing the code that stops the observing to create more defined meshes so the observer is always observing, and the same issue occurs.

It is important to note that the memory is not increasing as we are discovering new spaces. Essentially, I can stand up and not move the head by a bit, and the memory will still keep increasing.

Maybe I'm missing something, and if so please show me , but I feel there is some sort of bug in the SpatialMapping functionnality.

Thanks!

Patrick · July 2016

yeah...there's a slow leak. ``

I'll get a change ready to send to git, but in the meantime here's a preview of what I'm going to change:

In SpatialMappingObserver.cs@SurfaceObserver_OnSurfaceChanged

The remove case:

 case SurfaceChange.Removed:
                    // Always process surface removal events.
                    if (surfaces.TryGetValue(id.handle, out surface))
                    {
                        RemoveSurfaceObject(surface);
                        surfaces.Remove(id.handle);
                    }
                    break;

And in SpatialMappingSource I will add the function:

    /// <summary>
            /// When we aren't using a surface object any more we need to clean
            /// up the cached objects we made for the surface.
            /// </summary>
            /// <param name="surfaceObject">The surface we aren't using anymore.</param>
            protected void RemoveSurfaceObject(GameObject surfaceObject)
            {
                surfaceObjects.Remove(surfaceObject);
                MeshFilter filter = surfaceObject.GetComponent<MeshFilter>();
                surfaceObjectMeshFilters.Remove(filter);
                MeshRenderer surfaceMeshRenderer = surfaceObject.GetComponent<MeshRenderer>();
                surfaceObjectRenderers.Remove(surfaceMeshRenderer);
                Destroy(surfaceObject);
            }

Louis · July 2016

I actually found out what was the cause of the leak. I was still using version 5.4.0b16-HTP. When I updated to the .24 version, the leak was gone. I'm not sure what caused the leak precisely, but at least it's fixed now. Thanks for you help.

Patrick · July 2016

fun... looking.

Patrick · July 2016

yeah...there's a slow leak. ``

I'll get a change ready to send to git, but in the meantime here's a preview of what I'm going to change:

In SpatialMappingObserver.cs@SurfaceObserver_OnSurfaceChanged

The remove case:

 case SurfaceChange.Removed:
                    // Always process surface removal events.
                    if (surfaces.TryGetValue(id.handle, out surface))
                    {
                        RemoveSurfaceObject(surface);
                        surfaces.Remove(id.handle);
                    }
                    break;

And in SpatialMappingSource I will add the function:

    /// <summary>
            /// When we aren't using a surface object any more we need to clean
            /// up the cached objects we made for the surface.
            /// </summary>
            /// <param name="surfaceObject">The surface we aren't using anymore.</param>
            protected void RemoveSurfaceObject(GameObject surfaceObject)
            {
                surfaceObjects.Remove(surfaceObject);
                MeshFilter filter = surfaceObject.GetComponent<MeshFilter>();
                surfaceObjectMeshFilters.Remove(filter);
                MeshRenderer surfaceMeshRenderer = surfaceObject.GetComponent<MeshRenderer>();
                surfaceObjectRenderers.Remove(surfaceMeshRenderer);
                Destroy(surfaceObject);
            }

Louis · July 2016

I've tried the fix you've made and it seems to work with the Holograms 230: Spatial mapping tutorial when all settings are left untouched. Although, when I change the frequency of mesh updates from 3.5 seconds to lower (I've put 2), the memory stills grows. The code you've made essentially does the same thing as the Cleanup() function in SpatialMappingSource.cs, but for one surface instead of all of them, and the Cleanup() function doesn't seem to release any memory at all.

Patrick · July 2016

What are you using to detect the leak? I'm using Visual Studio's built in profiler. It might be missing something, but it definitely helped me identify the source of the leak yesterday. After the fix above memory usage was steady over several minutes. There might be another leak, and I'm hoping the tool you are using can help point me in the right direction.

Louis · July 2016

I used the HoloLens Portal performance tool (system performance tab), so nothing fancy at all. And I could see the memory drop over time (it didn't take long to see more than 40 MB than at the start of the application, maybe 1 or 2 minutes). Also, I don't know if you are testing on the actual HoloLens or on the emulator, but the bug didn't happen on the emulator.

Patrick · July 2016

okay, that helps. Back to looking.

Patrick · July 2016

@Louis at first I did see the memory climb. It leveled out pretty quickly though. I used the windows performance analyzer to analyze the memory usage of the system while the surface mesh update time was set to 1 second. I honestly don't see a leak (after I made the fix above).

Louis · July 2016

Ok, I've done further investigations. Here are the steps to reproduce the leak:

Download the Holograms 230: Spatial mapping tutorial (Make sure we have the latest version).
Open the "Completed" project in Unity.
Change the SpatialMapping asset so the Triangles per cubic meter is 1000 and the time between updates is 1 second.
Change the SpatialProcessing asset so the Scan Time is 600 seconds (We will wait 10 minutes to let the memory grow, then the scanning should stop, and the memory should stop increasing)
Make the necessary changes to the unity project in order to build the solution (I followed instructions in the tutorial: player settings, quality settings, build settings).
Build the project and open the visual studio solution.
Here, I made sure that the changes you made were in the SpatialMappingObserver and SpatialMappingSource. They were.
Build the solution on the lens, in release mode, x86 platform.
Open the Microsoft HoloLens Portal and go to the System Performance tab.

Here are the screen shots I took when doing the tests:

As you can see, apart from the first minute of run where the memory decreases, I lose around 80 MB every 2 minutes. Then, after the 10th minute, when the observer is stopped, the memory stops growing, and remains pretty stable for 4 minutes. And again, I haven't walked during the testing. I remained at my desk and looked pretty much at the same direction( my screen ).

If you can't reproduce the leak, then I don't know what I am doing wrong

Patrick · July 2016

Thanks! I'll follow these steps. I was testing in the HoloToolkit version of the scripts, so maybe something is still lurking in Holograms 230 that I missed.

Louis · July 2016

I have the same problem in my own application though, which is using the holotoolkit. I was only using the Holograms 230 tutorial to isolate every other aspect of my application, to make sure it wasn't me who introduced the leak.

Patrick · July 2016

Well this is going to be fun. You are clearly experiencing a leak. I'm not seeing it.

Have you taken any other steps? Made any other changes to the code? It would be helpful if you ran with memory tracking enabled. In Visual Studio press alt+f2, and enable memory usage. (that will turn off everything else). There's a blue start button in the same panel, press it. Then while the app runs, press the 'take snapshot' button every so often. Once you have a few snapshots over several minutes you can press the stop button. You'll be able to compare snapshots to see which stacks are allocating memory. This should help us find the leak and stop it.

I have a 1 second refresh time.'
TPCM is set to 1000.
Scan time is set to 600.
The app exited after the 10 minutes (I never got to the see the planes, a different bug for a different time).

The memory graph is like:

Before I start the app

Just after I see a little spatial mapping:

then a minute:

another minute:

another another minute (hopefully these are ordered right, but you'll see the point):

Louis · July 2016

@Patrick I can't to figure out how to start the diagnostic session on the HoloLens. When I press start, it fails with that error:

Louis · July 2016

And btw, I had that same bug you did where the app just closed by itself. It's like VS closes the app or something like that, as I experienced that bug very often when I'm debugging, but never when the app is runned without the link to VS (when I start it from the holoLens, for instance). Maybe it has something to do with the new version of VS2015 (update 3), as I don't remember having that bug before updating to update 3.

Patrick · July 2016

Did you reboot your HoloLens after getting that message? I'm guessing you did, but I have to ask. If so, what if you only enable CPU usage (Make sure that GPU Usage is not checked...)?

Patrick · July 2016

or...another idea...can you zip up assets and project settings from your planetarium project and post it here? The easiest explanation is we have a difference between our projects. In fact... here's mine, you can double check that I didn't miss a step.

Louis · July 2016

I actually found out what was the cause of the leak. I was still using version 5.4.0b16-HTP. When I updated to the .24 version, the leak was gone. I'm not sure what caused the leak precisely, but at least it's fixed now. Thanks for you help.

Patrick · July 2016

Oh, thank goodness.

Observer memory growing indefinitely

Best Answers

Answers