The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
SpatialUnderstanding freezes app when finalizing room mesh
Reeley
✭✭✭
Hey guys 
i understand that the spatial understanding is doing some heavy stuff when finalizing my scan, but is it normal that the whole application freezes for a moment? Cant this be done asynchronously? Or shouldnt my application freeze and im having some error in my code?
Best Answers
-
Options
trzy
✭✭✭
@Reeley I think I've found a workaround to the temporary lock-up that occurs when initializing the solver and placing objects: run it in a separate thread. SpatialUnderstanding is not thread-safe but is independent of Unity. You can certainly run it in another thread of your choosing and poll for completion in an Update() function in the Unity main thread.
I've confirmed it works for solver initialization. Look at System.Threading.Thread to implement a simple asynchronous task mechanism. Unfortunately, this will only work in the Unity editor. UWP requires Windows.System.Threading.ThreadPool.RunAsync() to be used: https://forum.unity3d.com/threads/how-to-get-around-the-threading-limit.249742/
Querying for object placement positions can take a lot of time as well, making it a great candidate for this sort of asynchronous off-loading. However you have to be careful if you intend to use Unity Objects in another thread. I've never done it myself but there appear to be resources online. Alternatively, you can reserve placements using the spatial understanding API in your worker thread and then once completed, instantiate GameObjects in the main Unity thread.
For an example of how I call Solver_Init() in a separate thread, see: https://github.com/trzy/hololens/blob/master/Game-Crane/Assets/Scripts/PlayspaceManager.cs
Specifically, the Update() function, State.WaitingForPlacementSolverInit case.
You will not be able to build this project yourself because I cannot push copyrighted assets to Github but this should serve as a useful starting point.
Bart
http://trzy.org5 -
Optionstrzy
✭✭✭
I actually now use multi-threading and it works great. I do it different than Microsoft does, though. In their examples, they launch an asynchronous task (each of which is scheduled in one of N worker threads that are made available by Windows) for each query and then wait until all are complete. That's fine and probably how most people would want to do it but in my case, I have some queries that depend on the results of others and prefer to just serialize everything in a single worker thread.
You can see how I do it here:
https://github.com/trzy/hololens/blob/master/Game-Helicopter/Assets/Scripts/TaskManager.cs <-- This script creates a long-running background thread that sleeps and waits for work items to come in on a queue. Once each item is processed, it can return a callback that is enqueued on a completion queue, which is to be executed in the main thread. If autoRunCompletionCallbacks is enabled, you'll see that there is a LateUpdate() function that will check this queue and empty it out, running the callbacks each time.
Here's an example of how I perform a query (actually, in this case, I am placing up to 3 objects and therefore attempt 3 queries):
===
PlayspaceManager pm = PlayspaceManager.Instance; Vector3 agent1Size = Footprint.Measure(agentPrefab1); float agent1Width = Mathf.Max(agent1Size.x, agent1Size.z); for (int i = 0; i < 3; i++) { TaskManager.Instance.Schedule( () => { Vector3 position; if (pm.TryPlaceOnPlatform(out position, 0.25f, 1.5f, 1.5f * agent1Width)) { return () => { NavMeshHit hit; if (NavMesh.SamplePosition(position, out hit, 2, NavMesh.AllAreas)) position = hit.position; Instantiate(agentPrefab1, position, Quaternion.identity); }; } return null; }); }===
PlayspaceManager is my own component but it basically is just a wrapper atop Spatial Understanding. You'll notice that there is a callback being passed to TaskManager::Schedule(). This is what gets executed in the background thread. It may not access any Unity API functions but it can call SpatialUnderstanding queries (that's all TryPlaceOnPlatform() is). If the query is successful, I want to instantiate an object there but I can only do so in the main thread. So you'll see that I return a completion callback. The task scheduler pushes that back to the main thread and executes it there.
Bart
http://trzy.org5
Answers
I've noticed this happening as well but haven't really dug into it too deeply. I think it's somewhat unavoidable. The way the HoloToolkit wrapper around spatial understanding is structured, no asynchronous mode is provided. It could probably done but would require extensive modification of the HoloToolkit.
I'd also be curious to know if there are any workarounds in the near term.
Bart
http://trzy.org
@trzy thanks for your reply. did you ever try to use spatialunderstanding on a big room? because i cant get it bigger than about 10x10m
No, unfortunately I haven't tried that yet. The rooms I usually scan are a bit smaller (but not by a lot). Have you tried digging through the source code to see if there are some hard-coded limits?
Bart
http://trzy.org
yes and there is one variable called ScanSearchDistance which gets passed to the SpatialUnderstandingDll, but increasing it didnt make a difference
@Reeley I think I've found a workaround to the temporary lock-up that occurs when initializing the solver and placing objects: run it in a separate thread. SpatialUnderstanding is not thread-safe but is independent of Unity. You can certainly run it in another thread of your choosing and poll for completion in an Update() function in the Unity main thread.
I've confirmed it works for solver initialization. Look at System.Threading.Thread to implement a simple asynchronous task mechanism. Unfortunately, this will only work in the Unity editor. UWP requires Windows.System.Threading.ThreadPool.RunAsync() to be used: https://forum.unity3d.com/threads/how-to-get-around-the-threading-limit.249742/
Querying for object placement positions can take a lot of time as well, making it a great candidate for this sort of asynchronous off-loading. However you have to be careful if you intend to use Unity Objects in another thread. I've never done it myself but there appear to be resources online. Alternatively, you can reserve placements using the spatial understanding API in your worker thread and then once completed, instantiate GameObjects in the main Unity thread.
For an example of how I call Solver_Init() in a separate thread, see: https://github.com/trzy/hololens/blob/master/Game-Crane/Assets/Scripts/PlayspaceManager.cs
Specifically, the Update() function, State.WaitingForPlacementSolverInit case.
You will not be able to build this project yourself because I cannot push copyrighted assets to Github but this should serve as a useful starting point.
Bart
http://trzy.org
Wow thanks for this detailed answer i will try that and post my results here
I actually now use multi-threading and it works great. I do it different than Microsoft does, though. In their examples, they launch an asynchronous task (each of which is scheduled in one of N worker threads that are made available by Windows) for each query and then wait until all are complete. That's fine and probably how most people would want to do it but in my case, I have some queries that depend on the results of others and prefer to just serialize everything in a single worker thread.
You can see how I do it here:
https://github.com/trzy/hololens/blob/master/Game-Helicopter/Assets/Scripts/TaskManager.cs <-- This script creates a long-running background thread that sleeps and waits for work items to come in on a queue. Once each item is processed, it can return a callback that is enqueued on a completion queue, which is to be executed in the main thread. If autoRunCompletionCallbacks is enabled, you'll see that there is a LateUpdate() function that will check this queue and empty it out, running the callbacks each time.
Here's an example of how I perform a query (actually, in this case, I am placing up to 3 objects and therefore attempt 3 queries):
===
PlayspaceManager pm = PlayspaceManager.Instance; Vector3 agent1Size = Footprint.Measure(agentPrefab1); float agent1Width = Mathf.Max(agent1Size.x, agent1Size.z); for (int i = 0; i < 3; i++) { TaskManager.Instance.Schedule( () => { Vector3 position; if (pm.TryPlaceOnPlatform(out position, 0.25f, 1.5f, 1.5f * agent1Width)) { return () => { NavMeshHit hit; if (NavMesh.SamplePosition(position, out hit, 2, NavMesh.AllAreas)) position = hit.position; Instantiate(agentPrefab1, position, Quaternion.identity); }; } return null; }); }===
PlayspaceManager is my own component but it basically is just a wrapper atop Spatial Understanding. You'll notice that there is a callback being passed to TaskManager::Schedule(). This is what gets executed in the background thread. It may not access any Unity API functions but it can call SpatialUnderstanding queries (that's all TryPlaceOnPlatform() is). If the query is successful, I want to instantiate an object there but I can only do so in the main thread. So you'll see that I return a completion callback. The task scheduler pushes that back to the main thread and executes it there.
Bart
http://trzy.org
Im currently not developing for hololens anymore, but this looks AWESOME, when i get back into developing for hololens i will try it out!
Thanks for your engagement!