Editing the Spatial Mesh

ccyuen

Would it be possible to export the Spatial Mesh .obj file from the device portal and load it into a 3D modeling software to then be edited? (i.e. smoothen out edges or make the floor more flat) I'm planning on creating a 2D plane based on the spatial mesh's rendering of the floor then dividing the plane up into a grid so I can setup some sort of coordinate system with walkable and non walkable nodes. If I could edit the spatial mesh to make it more uniform, it'll make the plane creating script more accurate and consistent when analyzing the spatial mesh's bounds.

Thanks.

jbienzms

@ccyuen, perhaps you could rephrase your question? @james_ashley's answer seems to be correct. You asked if you could export the spatial mesh from the portal and edit it using external software. He showed you how to export it. Many 3D packages readily support editing of OBJ. Is there something else you are wanting to do? e.g. are you asking if it's possible to load that edited OBJ back in and use it INSTEAD of the mesh dynamically generated by the HoloLens? If so, you can use ObjectSurfaceObserver for this, but you may have to find a way to manually align (rotate, translate) it to fit your room.

jbienzms

@ccyuen it's important to keep I'm mind that there are both hardware systems and software systems working in concert on the HoloLens. The spatial mesh that you see in Unity is a Unity reconstruction of data streamed in from the HPU and into the OS. This "hardware data" can only be faked in the Device Portal under simulation. You basically capture a space and then you can use that space again later for simulation. The simulation can either be done on the device or in the emulator. The file format for this simulation data (as far as I know) is not documented and is not easily edited.

However, there is still the software side. Remember that the spatial mapping mesh is generated in Unity based on the data received from the OS. As far as I know, all interactions in Unity are based on this software mesh. So, if you replace the software mesh you can replace the interactions. This is easily done through the ObjectSurfaceObserver. The only catch, again, is that ObjectSurfaceObserver just loads any Unity mesh stored in the project instead of asking for the data from the OS / HPU. The mesh will load as if it is the surface mesh, but it will not likely be aligned with your environment. That's why you may need to find a way to rotate and position this loaded mesh to align it with your physical world.

Again, remember that this is just loading a "canned" mesh and effectively bypassing the data received from the hardware. This mesh will never update unless you edit it yourself, and obviously it won't match any other location.

Does that answer your questions?

james_ashley

@ccyuen,

On the 3D View screen in the Device Portal, go down to Spatial Mapping and click the save button to get an .obj.

ccyuen

@james_ashley yes I already understand how that works. My question was asking if I can edit that file.

jbienzms

@ccyuen, perhaps you could rephrase your question? @james_ashley's answer seems to be correct. You asked if you could export the spatial mesh from the portal and edit it using external software. He showed you how to export it. Many 3D packages readily support editing of OBJ. Is there something else you are wanting to do? e.g. are you asking if it's possible to load that edited OBJ back in and use it INSTEAD of the mesh dynamically generated by the HoloLens? If so, you can use ObjectSurfaceObserver for this, but you may have to find a way to manually align (rotate, translate) it to fit your room.

ccyuen

@jbienzms Yes sorry, I mean to take that .obj file into a 3D modeling software, edit it and then export it to use. For example I would take a spatial mesh object of a room and edit it so that the floor is completely flat and use it in my unity project instead of the one stored in the HoloLens. Is this still possible through the ObjectSurfaceObserver? Or even if it is possible I foresee some bugs or issues that will pop up when trying to do this.

jbienzms

@ccyuen it's important to keep I'm mind that there are both hardware systems and software systems working in concert on the HoloLens. The spatial mesh that you see in Unity is a Unity reconstruction of data streamed in from the HPU and into the OS. This "hardware data" can only be faked in the Device Portal under simulation. You basically capture a space and then you can use that space again later for simulation. The simulation can either be done on the device or in the emulator. The file format for this simulation data (as far as I know) is not documented and is not easily edited.

However, there is still the software side. Remember that the spatial mapping mesh is generated in Unity based on the data received from the OS. As far as I know, all interactions in Unity are based on this software mesh. So, if you replace the software mesh you can replace the interactions. This is easily done through the ObjectSurfaceObserver. The only catch, again, is that ObjectSurfaceObserver just loads any Unity mesh stored in the project instead of asking for the data from the OS / HPU. The mesh will load as if it is the surface mesh, but it will not likely be aligned with your environment. That's why you may need to find a way to rotate and position this loaded mesh to align it with your physical world.

Again, remember that this is just loading a "canned" mesh and effectively bypassing the data received from the hardware. This mesh will never update unless you edit it yourself, and obviously it won't match any other location.

Does that answer your questions?

jbienzms

Oh, P.S. if what you're looking for is a normalized smoothed mesh of the room you probably want to check out Spatial Understanding. This is dynamically generated and can be used for any room. It's a little complicated to use, but the mesh produced is very clean, it's water tight, and you can execute queries against it.

Spatial Understanding is part of the toolkit. You’ll find it here:
 
https://github.com/Microsoft/HoloToolkit-Unity/tree/master/Assets/HoloToolkit/SpatialUnderstanding
 
And there is a sample scene you can play with here:
 
https://github.com/Microsoft/HoloToolkit-Unity/tree/master/Assets/HoloToolkit-Examples/SpatialUnderstanding
 
You can also check out a video of someone trying that sample scene here:

https://www.youtube.com/watch?v=SKpkYHc542o&feature=youtu.be

ccyuen

Yes @jbienzms, Thank you so much

ccyuen

Also @jbienzms how would I go about changing the Spatial Understanding scripts so that it wouldn't be dynamic? I don't want to scan the same room everytime I deploy the app, I want it to load the spatial mesh data from the OS/HPU and convert that into the Spatial Understanding mesh. I am trying to integrate the functionality of the Spatial Understanding scripts into my project.

jbienzms

@ccyuen for saving the SU Mesh check out this code here. Specifically look at the SaveSpatialUnderstanding scene.

LieBe

@james_ashley said:
@ccyuen,

On the 3D View screen in the Device Portal, go down to Spatial Mapping and click the save button to get an .obj.

I am sorry to disturb you. I want get a better(finer grid) spatial mapping.obj files from hololens. DO you know how to get it? BY the way,DO you know the max value of the TPCM(triangles per cubic meters),and TPCM default value is 500,

CristinavReynel

@LieBe said:

@james_ashley said:
@ccyuen,

On the 3D View screen in the Device Portal, go down to Spatial Mapping and click the save button to get an .obj.

I am sorry to disturb you. I want get a better(finer grid) spatial mapping.obj files from hololens. DO you know how to get it? BY the way,DO you know the max value of the TPCM(triangles per cubic meters),and TPCM default value is 500,

As far as I know the maximum is about 2000 triangles per cubic meter, you can put more but won't change anything.