Holocamera!!

YairYepez · April 2016

Hello,

QUESTION: How as a developer can I make one of these HoloCameras?

In all the Hololens live demo videos, like keynotes and TED talks there is always a DSLR camera attached to a Kinect One or Hololens or other device that captures the Holoexperience of the Hololens user. You can see that camera here in theis video from Build 2016 at minute 10:24 :

https://www.youtube.com/watch?v=mM1P41qeVGs&nohtml5=False

QUESTION: How as a developer can I make one of these HoloCameras? Thanks

AlexD · April 2016

Not sure how you can build one, but you can capture whatever the user is seeing using the Mixed Reality Capture in the device portal.
You can also film the person at the same time with an external camera, then merge the 2 videos in a PIP style video.
I was able to do it with only amateur hardware and software so I am pretty sure it's possible to get a very nice looking result.

andrewpdupuis · April 2016

I've been thinking about the same problem. One way I think you could do it is to use a Windows Phone running another copy of your unity app over UNET. Though you wont be able to move the camera (you'll have to hardcode the position or provide some way of aligning the position with real world) you could still set up a shot and capture pretty well.

Add a gameobject to the unity project that captures from the device camera (http://blog.falafel.com/showing-device-camera-unity/) that only show up if you're on a windows phone (Use a compiler directive #if WINDOWS_PHONE). Set the Unity virtual camera's field of view to match the phone's camera.

Though certainly not very flexible, I would think this would allow you to screen capture the phone and get a decent "holocamera".

Let me know if you find out a way of making a camera for this stuff!

YairYepez · April 2016

@AlexD

@AlexD said:
You can also film the person at the same time with an external camera, then merge the 2 videos in a PIP style video.
I was able to do it with only amateur hardware and software so I am pretty sure it's possible to get a very nice looking result.

Do you have a sample video of this merged footage?

I still haven't received my Hololens DevKit so I am getting ready to have a solution to this problem by then. This #Holocamera would be very very useful for developers!!!

Thanks!

YairYepez · April 2016

@andrewpdupuis said:

I've been thinking about the same problem...Add a gameobject to the unity project that captures from the device camera (http://blog.falafel.com/showing-device-camera-unity/) that only show up if you're on a windows phone (Use a compiler directive #if WINDOWS_PHONE). Set the Unity virtual camera's field of view
Let me know if you find out a way of making a camera for this stuff!

This is a good option but I think that attaching a Kinetic One to any DSLR camera can do the same trick with less trouble. A Raspberry Pi running Windows 10 or a Surface Pro added to the mix might do the Unity processesing and paring possible. Basically attaching a rudimentary Hololens (Windows 10 computer + Kinect One) to a camera and the result might happen. Just a thought on how to build this Holocamera.

Will let you know if I find something. Thanks for the help. Wish Microsoft would release some info on How to build on of this Holocamera!!! #Holocamera

YairYepez · April 2016

It would be extremely super useful for Hololens developers to beable to build this #Holocamera
Microsoft Staff, any thoughts?

-Yair

Patrick · April 2016

The photo booth we had at build (the source of the images of people next to the Sun) we are working on making open source. This is a different system from what is used for on stage demos. One major difference is the photo booth only captures still images. Adding video to the photo booth is under consideration and certainly possible, though there are some specific challenges with synchronizing the camera poses from the devices. I'm not aware of any plans to make the system used for on stage demos available outside of Microsoft.

HoloSheep · April 2016

Making the source of the photo booth solution available would be awesome. Any rough timeline on when that might happen?

Another capture device that might be interesting to try and integrate into a solution for this kind of merged display is a Project Tango tablet, which also works with Unity but I am not sure how difficult it would be to get their spatial data to play nice with a HoloLens project and the HoloLens spatial data.

Patrick · April 2016

I'm fairly hesitant to share timelines. I will say that the team who created the Photobooth has getting the project open sourced high on their priority list.

Getting the Tango and the HoloLens to sync...that would be cool to see. The challenge I foresee would be getting both devices to accurately agree on where they are in the physical world.

HoloSheep · April 2016

@Patrick said:
Getting the Tango and the HoloLens to sync...that would be cool to see. The challenge I foresee would be getting both devices to accurately agree on where they are in the physical world.

Maybe some form of world truth could be used to establish a common reference like three markers or identifiable images that could be locked onto.

Alternatively a less precise and more manual approach might be for the user to identify a few common corners on both worlds.

Once agreement is established on a common space, each device could report its own position in that space to the other.

YairYepez · June 2016

The Hololens companion app for Windows 10 turns out to be the best option to turn any Microsoft Surface + a pair of Hololens into a #Holocamera

Holocamera!!

Answers

Windows Holographic User Group Redmond

Windows Holographic User Group Redmond