Hololens depth sensor and technology?

pedro_orellana

I need Just the principe of the technology. Like if is structured infrared like kinect v1 or time of flight camera like kinect v2, or some totally diferent ( rgb stereo vision, etc ... ).

I know the limitation of range ( no more than 3 meter aprox ) and light ambient of the depth technology, but I need more info to have and idea of the 3D model quality.

HoloSheep

So are you asking about the Spatial Resolution that HoloLens is capable of?

james_ashley

@pedro_orellana ,

It's time-of-flight. A good friend checked the IR pattern for me.

Brekel

Yep, I looked at a running HoloLens with a Kinect v2 sensor, an Orbbec Astra Pro and a generic InfraRed camera.

I didn't see any speckle (or other) patterns commonly associated by a Structured Light sensor.

Interestingly enough Kinect v2 and Orbbec showed some minor flickering lights on the HoloLens when looking directly at it but didn't see any of it's projected light. It seems these operate in different IR spectrums and don't interfere with each other.

The InfraRed camera (which sees a wider spectrum than Kienct v2 and Orbbec's) showed the HoloLens emitting the scene.
It looked like it was illuminating similarly to other Time of Flight cameras by flooding the scene with very short burst of light.

Jesse_McCulloch

So, I don't really have a great understanding of this tech, but I know the HoloLens has a tough time mapping a dark room. As such, would an infrared light source in the room help or hinder?

Jarrod1937

@Jesse_McCulloch said:
So, I don't really have a great understanding of this tech, but I know the HoloLens has a tough time mapping a dark room. As such, would an infrared light source in the room help or hinder?

A dark room is tougher? I'd be surprised, since I think the sensor is providing it's own illumination. As such, a brighter area may have issues, not dark.

Jesse_McCulloch

Yeah, I was kind of surprised as well. But I was in my bedroom with the lights off to try and watch a movie, and it could not map the room sufficiently enough, so it put it into limited use mode...

Jarrod1937

If it is similar to the Kinect for the Xbox One, then here is a really in-depth rundown of the tech:

http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6964815

The fact that it is time of flight in the first place is impressive. Say the round trip distance from the source, to the object, back to the sensor is 3 meters. The speed of light is 299,792,458 m/s, so that is 3/c = 10.0069 nanoseconds, which is 9.9 MHz timing, for a single photon ray/pixel that the sensor is capturing. Of course you need far higher sampling to differentiate features smaller than 3 meters. If they achieve a 5 cm resolution that is 0.05 m / c = 0.16678 ns or 5.99 GHz timing, pretty crazy.

HoloSheep

@Jesse_McCulloch there is mention in the documentation under Handling Tracking errors that: "Sometimes, the device's sensors are not able to figure out where the device is. This can happen if the room is dark, or if the sensors are covered by hair or hands, or if the surroundings do not have enough texture."

Which would indicate that the HoloLens, like many other fusion sensor devices such as Project Tango tablet, also relies on visible light cameras like the 4 environment understanding cameras mentioned in the HoloLens hardware spec, along with the depth camera (which leverages the IR light emitters) for area mapping and understanding.

Jarrod1937

Ah thanks @HoloSheep, I didn't consider the visual wavelength cameras as well. Now the darkness issue makes sense.

ContextVR

A follow-up question to the HoloLens team: is it possible to switch from RGB camera to IR cameras to enable at least some degree of positional tracking in the dark?