Hello everyone.
The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Options
VR Camera capture to holograms?
Can anyone provide some guidance on what it takes to leverage a VR camera and convert it for use in the hololens? Additionally:
Are there any cameras that work better than others?
What formats are supported?
What software is needed for conversions?
It seems like most of the examples are based on content that is created from software from scratch. I would like to take real life things and turn them into holograms.
1
Answers
If you consider "VR camera" to be a camera that creates either 360 video or stills.. there is at least one other thread on the subject that you may find of interest.
For 360 recording there are definitely a very wide range of cameras and quality on the market. They range from a few hundred bucks for a reasonably decent low-end camera like the Ricoh Theta S to potentially severally hundreds of thousands of dollars for something like the Lytro Immerge, which I would expect to work better than say the Richo Theta S.
However, besides quality, I am not aware anything specific to look for that would make one camera more appropriate for recording 360 content intended for the HoloLens versus some other VR or AR platform. I say that partially because there is no specific 360 support for the HoloLens as of yet, it is all custom implementation of the general Unity and HoloLens APIs to come up with various 360 solutions.
To the best of my knowledge there has been no out of the box support for 360 content on HoloLens yet. The only example showcasing 360 content that I have seen that is downloadable as of yet is the HoloTour app.
The conversion workflow for processing 360 content is still evolving and many camera manufactures, software companies and pioneers have posted some of their approaches in blog posts and YouTube videos but I haven't seen anything specifically targeting a 360 to HoloLens workflow as of yet. So searching for 360 workflows appropriate to the 360 camera brand or budget range that you decide on is one place to start.
Typical holograms on the HoloLens by their nature are generated by 3D volumetric models and are not just projections of 2D content like 360 stills or video (as cool as those projections of 2D on the HoloLens are). So if your interest is in taking photographs and videos for the purpose of creating 3D models that area of specialty is called Photogrammetry and is a specialty unto itself. There are some open source and commercial software packages out there that can be used to process photos or video that you might look into.
Creating your volumetric models, whether that is with Photogrammetry, or traditional 3D software tools like Maja, 3DS Max, and Sketchup is one of the tasks required in order to assemble the content into a holographic experience on the HoloLens. Besides creating new content, you will also find lots of existing content available through sites like the Unity Asset Store or 3D Warehouse to name just a couple.
Also there are lots of interesting research projects going on in areas like the MSR Holoportation capture technique that provide examples of where the technology is heading.
Windows Holographic User Group Redmond
WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
WinHUGR YouTube Channel -- live streamed meetings
I use Kinect 2 for Windows, basically an Xbox One Kinect with a USB 3 adapter, on same Desktop PC for HoloLens Emulator. Cost under £200 from Amazon UK. For making 3D 360 degrees scans
This comes with masses of free software support, with ALL the source code to build you own version. Window 10 Insider build 14xxx onward support as this well and brings in with it Windows Store Apps 3D Scan and 3D Builder.
We are developing this camera: lightboxvr.io. It will be very suitable for the purpose of taking real life things and turn them into holograms as mentioned in the original question of this thread. Please message me if interested or contact us using the web form!
When you complete the final product I suggest you go into the sections marked "Projects" Start a project called
"Lightbox : a new kind of camera that generates holograms". Then you can explain the detail and interact with potential users.
If you have a crowd funded project then mention that
Hey, do you have any resources for embedding a video feed from Kinect into a hologram?
You would need to change from Ricoh to Kinect in 14393 Kinect just becomes a webcam device so maybe this would help
http://theta360.guide/blog/video,/unity/2016/10/16/video-file-on-unity-sphere.html
Its not really that different to the original code just that the Hologram would need to be imported into Unity as the GameObject
I don't have any idea how to implement this in Unity 5.6 for real
Ah, I guess I wasn't clear. It could really be from a Kinect or a 360 camera, I have access to both, but I was looking for a way to show a live video feed. Any idea about that?
The frame rate is killer, You cannot get the frame rate of 60FPS to prevent severe jarring and leaves you feeling sick on a VR or AR system.
Unity 5.5 CANNOT produce this, hence the new 5.6 player that needs a 4.4k stream and 60FPS, which Kinect cannot supply.
Kinect for Windows Sensor Components and Specifications
Code sample where failure takes place
private WriteableBitmap wBitmap;
wBitmap = new WriteableBitmap(...);
depth.Source = wBitmap;
wBitmap.WritePixels(...);...
Most people says they feel motion sickness at video at lower than 30 FPS on ALL VR/AR devices, especially on Gear VR its awful!.
Really good quote sums it up
I see! Thanks so much for the info!
I will look into it further on my own, since I don't need a 4k stream (I only want a small video hologram showing a different view or possibly a close-up).