The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
How to place an Hologram to a moving object?
I would like to give a holographic character to my toy car. The hologram should be placed on the car and be visible while the car moves around. How to do this?
Best Answers
-
Options
prenex
✭
Hei!
If you are ready to update the car itself with some sensors you can investigate possibilities of measuring its position with accurate-enough feedback sensors and stream the position data into the hololens through wifi or bluetooth maybe.
Also I see that Vuforia already supports the device. With it, you can place a marker on the car and use that as the reference for the moving objects. I have only mobile app development experience with Vuforia and things like ARTK though so I don't know about how much it works or lags. ARTK is free, more direct and maybe harder to use in your case, Vuforia has free versions for personal usage and supports unity and hololens as far as I know so you can play around with this by yourself.
Best regards,
Richard Thier
R&D / IT Expert - GrepInvent @ Grepton Zrt.5 -
Options
@prenex said:
Also I see that Vuforia already supports the device. With it, you can place a marker on the car and use that as the reference for the moving objects.
Unfortunately Vuforia for Hololens works different from other devices. It doesn't update smoothly like on mobiles, it pins it with the world anchoring system when it's detected, then it only updates like twice a second, so if the target is moving it jumps around.
5 -
Options
CurvSurf
✭✭
Microsoft MUST provide developers with the raw depth data stream. Then, they will try to put in real-time:
- Holograms onto moving objects
- Videos/photos onto existing object surfaces.
At the moment, you can extract only planar and large enough object surfaces by using the plane algorithm of HoloToolkit: You can put Holograms or videos/photos onto floor, wall, tabletop, etc. but not onto a curved object surface, e.g. spherical or cylindrical surface.
5
Answers
Unless the toy car is very large and moves slowly it probably is not doable. The spatial surfaces have a rather limited resolution and a small toy car would get at most few polys that would be unstable due to the car's topology. To complicate it more there is noticeable delay between the real world and when the surfaces are ready for the "user" (the app), so the character would always appear out-of-sync with the car if the car is moving rather quickly.
Hei!
If you are ready to update the car itself with some sensors you can investigate possibilities of measuring its position with accurate-enough feedback sensors and stream the position data into the hololens through wifi or bluetooth maybe.
Also I see that Vuforia already supports the device. With it, you can place a marker on the car and use that as the reference for the moving objects. I have only mobile app development experience with Vuforia and things like ARTK though so I don't know about how much it works or lags. ARTK is free, more direct and maybe harder to use in your case, Vuforia has free versions for personal usage and supports unity and hololens as far as I know so you can play around with this by yourself.
Best regards,
Richard Thier
R&D / IT Expert - GrepInvent @ Grepton Zrt.
Unfortunately Vuforia for Hololens works different from other devices. It doesn't update smoothly like on mobiles, it pins it with the world anchoring system when it's detected, then it only updates like twice a second, so if the target is moving it jumps around.
This is very good to know about at least!!!
I have also found performance things like this, but there are always a lot innovative lightweight solutions to most problems with which you can avoid things like this. One should also use up controlled environments I guess - for example if you know there are nothing green in a room , you can try to do some "hax methods" in image processing for finding "marking" green lights / leds as dots of the car with some very linear search for having better results as vuforia if it is this bad as you say... I was thinking about this latter thing for myself so why not share the idea...
@LeefromSeattle, @prenex
Microsoft MUST provide developers with the raw depth data stream. Then, they will try to put in real-time:
At the moment, you can extract only planar and large enough object surfaces by using the plane algorithm of HoloToolkit: You can put Holograms or videos/photos onto floor, wall, tabletop, etc. but not onto a curved object surface, e.g. spherical or cylindrical surface.
We still use Vuforia for quick, precise placement of content on the fly in a tradeshow atmosphere. Sometimes the collider from Hololens is a couple inches above/below the actual surface and we're in a huge booth, we tested the hololens 60' from the targets with things moving between them and it works. I don't child the objects to Vuforia targets, just when the target is detected it snaps the content to it and pins it, then I remove the target from the area. I was using tap to place but anything that can go wrong at a tradeshow does with hundreds of people running around.
I tried opencv with face/hand tracking but it doesn't run fast enough. In the past I've tried tracking color similar to how the tracking.js algo works, and it works but it's really slow in Unity to iterate over a texture's pixels, even with their "fast" method getpixels32/setpixels32 on a desktop PC.
DX11 compute shaders in Unity could be a workable solution, but would take effort.
Hi
@prenex, @LeefromSeattle,@CurvSurf ,@Alex480
Thank you for your feedback! I used Vuforia and it doesn't work that well like @LeefromSeattle pointed out. Now I want to explore the sensors idea. I want o make something like that robot with the holographic interface:
https://www.youtube.com/watch?v=xnrHFV34PfM
All ideas are welcome!
Thank you
Hi,
I'm struggling with Vuforia for a while. Can you please tell me what do you mean by "it doesn't work that well"? Have succeed in putting Vuforia and Hololens to work together anyhow? I wasn't able to draw anything on top of the marker so far and tried all tricks and advices from this forum.
Please, let me know what you did.
Thanks a lot!
Zona
@LeefromSeattle can you describe how you were able to run opencv on hololens?
Hi @zona
I followed step by step the Vuforia for Hololens sample. 'It doesn't work that well ' means that when I move, the teapot hologram moves as well, it does not stay at the precise position I would like it, and it seems to float high above the plane I want it to stay. If anyone knows how to work around these issues, I would appreciate some insights! It might be to the size and position/rotation of the image marker I use?
Thanks!
Hi again @zona,
Check if you have the correct Player Settings namely 'Virtual Reality Supported'. I just upgrade Unity and this setting was modified, I needed to add the Windows Holographic in the Virtual Reality SDKs. Now it runs again, though with the same problems I stated before,( moving and floating).
Hope it helps!
Vuforia "pinning" an object is optional. In many scenarios the target doesn't move, just the user. So once the target has been identified it can be pinned to the world and the HoloLens native tracking takes over. But this feature can be turned off.
Having said that, I agree with everyone above that using an ImageTarget is probably not going to be sufficient. Especially depending on how fast the car is moving. The video shown above of the robot is probably the best approach for this need. I talked to a member of the team that did this demo and he said they used bright red and green LEDs in a known configuration then they used computer vision to track it. He did not indicate if they used OpenCV or something custom for the tracking.
I have not personally attempted to get OpenCV running on HoloLens yet, but I did start some research on how to do it. I was going to attempt to get Emgu CV running there since they say they support Unity.
http://www.emgu.com/wiki/index.php/Main_Page
If you get this working I would be very interested to hear about your results.
Our Holographic world is here
RoadToHolo.com WikiHolo.net @jbienz
I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.
@LeefromSeattle can you describe how you were able to run opencv on hololens?
@xhensiladoda,
this post is talking about opencv build (latest) on hololens.
https://forums.hololens.com/discussion/comment/11532/#Comment_11532