Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Hololens with Leap Motion+Vuforia

ZenZen
edited August 2016 in Projects

This is part of my summer research, and during the summer I have been trying to integrate the Leap motion API and make it work with Hololens. The purpose of this would be accurate hand occlusion/interact with the environment etc. This is what I have for now:

The goal for my reserach is to use Vuforia to do object/image recognition, and generate 3d content on a real world object. For example generate a live baby on a baby mannequin in medical research, and interact it with using leap motion.

Right now the leap motion does not fit perfectly with my hand because the coordinate system does not really match. Welcome every suggestion on coordinate system alignment and mroe!

Tagged:

Comments

  • very cool!
    Where did you place the leap physically? if you're able to mount it, there should be a clean path to translate the coordinates I think,
    Can you elaborate on how you got Vuforia running on the device and say something about the experience?

    thanx!!

  • BrekelBrekel ✭✭

    Nice!

    I've been wanting to try this for myself but haven't gotten around to it yet.
    Great to see you got it working!

    I'm assuming you're streaming the Leap Motion in over WiFi right?

  • @tzurs I taped it on top of the hololens.

    Vuforia has their support for Hololens now, so I just use them. It works with some minor issues. For example I can't take videos of photos since the vuforia is taking porcess of the webcam.

  • @Brekel Yep through a wifi network.

  • Very cool work. I love the use of the occlusion mask with the hands to enhance presence. Have you thought about expanding the work to be untethered?

  • Awesome job! On YouTube, you said you'd put the code on GitHub soon. Can't wait.

  • Well done! I just wondering what about putting the project on GitHub?

  • Awesome! Are you able to render leapmotion field of view as another camera space? I need to set up a more stable enviroment inside hololens spatial mapping for a 6dof controler (reflexive stick) for surgery guidance and mesh position.

  • @DrLampert said:
    Awesome! Are you able to render leapmotion field of view as another camera space? I need to set up a more stable enviroment inside hololens spatial mapping for a 6dof controler (reflexive stick) for surgery guidance and mesh position.

    This is great stuff! Are you visualizing your holograms through 3D viewer beta? I am working on similarly applying hololens in plastic surgery. Would love to chat sometime about what you are doing.

  • Wow! Great work Zen.

    How were you able to get so close to the holograms? Don't they disappear right at about the distance you'd be able to touch them?

  • Hi Zen, how did you stream from the leap to the hololens?

Sign In or Register to comment.