Can HoloLens device map real room stairs into a mesh?

Jimbohalo10

Can HoloLens device map real room stairs into a mesh?. You might think this is simple to put into a room for game, but this Tom Clancy's The Divison Beta - Stairs Bug (PC) shows how easily this can go wrong!.

So when making the HoloLens demo snippet, based this video, it occurred that "Can HoloLens see, in Augmented/Mixed reality, real stairs that can be climbed."

The demo does not want to look like the character bouncing off the invisible end of mesh or falling through the stairs.

Is there a example picture of room steps and matching mesh,somewhere as I don't want to put in the effort to find ends up like the video bug!!
Thanks

HoloSheep

@Jimbohalo10, I haven't tried something specific like this myself yet and I do not have a sample image and/or mesh to offer you but I would suggested there are a couple of factors you might want to take into account if you are developing an app to handle a scenario like stairs. Someone asked me a similar question at our recent hackathon because they noticed the rendering of the steps up to the stage at our venue were not coming out very clearly on the rendered mesh.

The real world lighting of the stairs in different environments may vary dramatically and could be a significant factor influencing how well the HoloLens can spatially map the stairs. If stairs are important to the experience you are creating you may want to have special logic and UI to step the user through the room scanning phase as well as some post processing of your mesh to specifically handle any stairs that you detect in your scene.

Your sample rate (TrianglesPerCubicMeter - unityengine.vr.wsa.surfacedata) settings will impact your accuracy with an inverse impact on performance when scanning with the SpatialMappingObserver. This value will need to be set high enough to capture stairs and the default setting of 500 is probably too low based on the experience of one of our hackathon participants.

So if your app detects issues with stairs you could switch to a special stair scanning mode and prompt the user to do some extra scanning of that area before the rest of the application gets underway.

druidmechanics

I can only speculate, but it should be generally possible. You would need to move enough for the HoloLens to scan the steps. The Origami tutorial includes a part in which you leave the mapping mesh visible, and you can try this on stairs.

The unknown for me is whether the HoloLens team have deliberately limited what constitutes a "room", such that at a point the steps will be truncated. I've seen this when playing Young Conker in a fairly large space -- at a point it was walled off, for what I assume to be imposed space limits. This makes sense to ensure a reliable game performance as well as to limit memory and performance concerns.

druidmechanics

Worn carpet might do fine if the dots are detectable.

If you have a video camera with night vision, try using it to see the dots. If not, look on YouTube to see the dot array. That may give you some ideas.

I should say that I've just assumed that this is the method by which HoloLens determines depth (this plus software); I haven't verified it.

HoloSheep

@Jimbohalo10, I haven't tried something specific like this myself yet and I do not have a sample image and/or mesh to offer you but I would suggested there are a couple of factors you might want to take into account if you are developing an app to handle a scenario like stairs. Someone asked me a similar question at our recent hackathon because they noticed the rendering of the steps up to the stage at our venue were not coming out very clearly on the rendered mesh.

The real world lighting of the stairs in different environments may vary dramatically and could be a significant factor influencing how well the HoloLens can spatially map the stairs. If stairs are important to the experience you are creating you may want to have special logic and UI to step the user through the room scanning phase as well as some post processing of your mesh to specifically handle any stairs that you detect in your scene.

Your sample rate (TrianglesPerCubicMeter - unityengine.vr.wsa.surfacedata) settings will impact your accuracy with an inverse impact on performance when scanning with the SpatialMappingObserver. This value will need to be set high enough to capture stairs and the default setting of 500 is probably too low based on the experience of one of our hackathon participants.

So if your app detects issues with stairs you could switch to a special stair scanning mode and prompt the user to do some extra scanning of that area before the rest of the application gets underway.

druidmechanics

I can only speculate, but it should be generally possible. You would need to move enough for the HoloLens to scan the steps. The Origami tutorial includes a part in which you leave the mapping mesh visible, and you can try this on stairs.

The unknown for me is whether the HoloLens team have deliberately limited what constitutes a "room", such that at a point the steps will be truncated. I've seen this when playing Young Conker in a fairly large space -- at a point it was walled off, for what I assume to be imposed space limits. This makes sense to ensure a reliable game performance as well as to limit memory and performance concerns.

Jimbohalo10

Thanks @HoloSheep and @druidmechanics this is the sort of problems I expected to encounter. Rough and worn carpet will only cause more problems. In principle there is a lot of things happening and processing limits, the layout to a single room. Probably Early times in the device development to make complex games. These processing problems are similar to trying to run the game on a tablet.

druidmechanics

Worn carpet might do fine if the dots are detectable.

If you have a video camera with night vision, try using it to see the dots. If not, look on YouTube to see the dot array. That may give you some ideas.

I should say that I've just assumed that this is the method by which HoloLens determines depth (this plus software); I haven't verified it.