Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Hololens with Leap Motion+Vuforia

ZenZen
edited August 2016 in Projects

This is part of my summer research, and during the summer I have been trying to integrate the Leap motion API and make it work with Hololens. The purpose of this would be accurate hand occlusion/interact with the environment etc. This is what I have for now:
https://www.youtube.com/watch?v=Yohj2ffvERA

The goal for my reserach is to use Vuforia to do object/image recognition, and generate 3d content on a real world object. For example generate a live baby on a baby mannequin in medical research, and interact it with using leap motion.

Right now the leap motion does not fit perfectly with my hand because the coordinate system does not really match. Welcome every suggestion on coordinate system alignment and mroe!

Tagged:

Comments

  • very cool!
    Where did you place the leap physically? if you're able to mount it, there should be a clean path to translate the coordinates I think,
    Can you elaborate on how you got Vuforia running on the device and say something about the experience?

    thanx!!

  • BrekelBrekel ✭✭

    Nice!

    I've been wanting to try this for myself but haven't gotten around to it yet.
    Great to see you got it working!

    I'm assuming you're streaming the Leap Motion in over WiFi right?

  • @tzurs I taped it on top of the hololens.

    Vuforia has their support for Hololens now, so I just use them. It works with some minor issues. For example I can't take videos of photos since the vuforia is taking porcess of the webcam.

  • @Brekel Yep through a wifi network.

  • Very cool work. I love the use of the occlusion mask with the hands to enhance presence. Have you thought about expanding the work to be untethered?

  • Awesome job! On YouTube, you said you'd put the code on GitHub soon. Can't wait.

  • Well done! I just wondering what about putting the project on GitHub?

  • Awesome! Are you able to render leapmotion field of view as another camera space? I need to set up a more stable enviroment inside hololens spatial mapping for a 6dof controler (reflexive stick) for surgery guidance and mesh position.

    https://www.youtube.com/watch?v=zC5097mA9f4

  • @DrLampert said:
    Awesome! Are you able to render leapmotion field of view as another camera space? I need to set up a more stable enviroment inside hololens spatial mapping for a 6dof controler (reflexive stick) for surgery guidance and mesh position.

    https://www.youtube.com/watch?v=zC5097mA9f4

    This is great stuff! Are you visualizing your holograms through 3D viewer beta? I am working on similarly applying hololens in plastic surgery. Would love to chat sometime about what you are doing.

  • Wow! Great work Zen.

    How were you able to get so close to the holograms? Don't they disappear right at about the distance you'd be able to touch them?

  • Hi Zen, how did you stream from the leap to the hololens?

Sign In or Register to comment.