The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Can Mixed Reality show you in a real car driving in a VR world?
It is for a project I am working on. If I can avoid a green screen solution that would be good, but first I need to get the prototype working with "something", then can do refinements.
Ideally I want to have a dual camera feed showing a 3d image of the car the person is in and out the windows / mirrors be the conventional "need for speed underground" or similar VR world. Will this tech do it? Any particular pointers on how?
Sure appreciate any help. This is not an "idle" question, you can see what I have done so far at www.vrdriversim.com/moki being the prototype page, and lots of related info on the various tabs and pages there... Thank you.
Best Answers
-
Options
hey Vince, I think you have the right idea in your comment "get the prototype working with something". I am going through the same thing.
Get your proof of concept up and running, doesn't need to be pretty it just needs to work.
I think the main thing to figure out what requires Mixed Reality vs Virtual Reality. Mixed reality you are implementing virtual objects (holograms) in a real world environment. you have about a 45 degree field of view to work with so you don't get that peripheral vision. So if you're looking for immersion like with a Vive/Oculus headset you won't get that.
What you can do however, is go a little bit of an AR route, maybe triggers when people look at certain parts of the car in the real world information pops up. Or create a virtual heads up display.
The way I would approach it is "What do I want to add to the real world to enhance this experience".
Hopefully that helps, feel free to ask more questions on the subject. The work you are doing looks interesting, I will keep an eye on this site and follow your progress.
5 -
OptionsVinceS ✭
Thank you for that useful insight. It is key to get visual and physical totally in sync, thus getting motion sickness outcomes to the low end. First is to match physical outcomes to control inputs, and it is a really solid body of work to do to get precise control of the simulator so what is input via the controls matches the sensation. THEN need to get a 100% match to associated VR feed, which I am exploring the smartest ways to do that. For physical realism, I will be datalogging a bunch of real world driving scenarios and matching that precisely on the simulator purely by steering, braking and accelerating, with road noise and bumps coming through in a realistic way.
For most, this would be pretty tricky work, but I have a really solid methodology to do it - we'll see how we go with that one, but high confidence. Nevertheless, minimum 400 hours work, and I notoriously underestimate tasks! From there I can start answering my own questions. I have just discovered Leap controllers and have one of them on the way as hand tracking from that combined with a vehicle interior overlay and a couple of artificial arms might be enough to get it fully believable. Where do I get a Mazda 2 overlay? Issue for another day!
When I get there, I will be able to figure out which bits really matter. It is bound to be different from what I am currently thinking! I just data entered all the costs associated with this thing and it adds up to $33k, about double what I was expecting and pretty well maxed me out, for now anyway. There is going to be a month of not much progress as my 57yo business partner of 22 years recently passed away, after a short illness, just as we were about to totally re-pitch the business, so this falls to me and is "a bit of a distraction".
My current solution to the OP q is to use green screening on the simulator - so painted windows and curtain behind. Not sure about the external mirrors, maybe totally VR generated. Had intended to use the Vive headset camera to show internals of the cab but it may be inadequate. So a couple of mobile ph cameras (you can get stand-alones) stuck on the headset in eye positions and can use Rift or Vive. All the joy of prototyping. But a lot of what I don't know is known, and accessing that is all I am hoping to do while I still have thinking time around this - ie before the self-imposed crisis arrives. Most likely, AR as it is currently being conceived is not the nature of the answer to the application. But love to hear any specific suggestions that could get me closer to real answers. Thank you for that.
0
Answers
hey Vince, I think you have the right idea in your comment "get the prototype working with something". I am going through the same thing.
Get your proof of concept up and running, doesn't need to be pretty it just needs to work.
I think the main thing to figure out what requires Mixed Reality vs Virtual Reality. Mixed reality you are implementing virtual objects (holograms) in a real world environment. you have about a 45 degree field of view to work with so you don't get that peripheral vision. So if you're looking for immersion like with a Vive/Oculus headset you won't get that.
What you can do however, is go a little bit of an AR route, maybe triggers when people look at certain parts of the car in the real world information pops up. Or create a virtual heads up display.
The way I would approach it is "What do I want to add to the real world to enhance this experience".
Hopefully that helps, feel free to ask more questions on the subject. The work you are doing looks interesting, I will keep an eye on this site and follow your progress.
Thank you for that useful insight. It is key to get visual and physical totally in sync, thus getting motion sickness outcomes to the low end. First is to match physical outcomes to control inputs, and it is a really solid body of work to do to get precise control of the simulator so what is input via the controls matches the sensation. THEN need to get a 100% match to associated VR feed, which I am exploring the smartest ways to do that. For physical realism, I will be datalogging a bunch of real world driving scenarios and matching that precisely on the simulator purely by steering, braking and accelerating, with road noise and bumps coming through in a realistic way.
For most, this would be pretty tricky work, but I have a really solid methodology to do it - we'll see how we go with that one, but high confidence. Nevertheless, minimum 400 hours work, and I notoriously underestimate tasks! From there I can start answering my own questions. I have just discovered Leap controllers and have one of them on the way as hand tracking from that combined with a vehicle interior overlay and a couple of artificial arms might be enough to get it fully believable. Where do I get a Mazda 2 overlay? Issue for another day!
When I get there, I will be able to figure out which bits really matter. It is bound to be different from what I am currently thinking! I just data entered all the costs associated with this thing and it adds up to $33k, about double what I was expecting and pretty well maxed me out, for now anyway. There is going to be a month of not much progress as my 57yo business partner of 22 years recently passed away, after a short illness, just as we were about to totally re-pitch the business, so this falls to me and is "a bit of a distraction".
My current solution to the OP q is to use green screening on the simulator - so painted windows and curtain behind. Not sure about the external mirrors, maybe totally VR generated. Had intended to use the Vive headset camera to show internals of the cab but it may be inadequate. So a couple of mobile ph cameras (you can get stand-alones) stuck on the headset in eye positions and can use Rift or Vive. All the joy of prototyping. But a lot of what I don't know is known, and accessing that is all I am hoping to do while I still have thinking time around this - ie before the self-imposed crisis arrives. Most likely, AR as it is currently being conceived is not the nature of the answer to the application. But love to hear any specific suggestions that could get me closer to real answers. Thank you for that.
Cr*p, just realised the REASON I asked this Q here in the first place was about the ACER Mixed reality headsets, eg https://www.cnet.com/au/products/acer-windows-mixed-reality-headset/preview/ to see if they were applicable. The discussion has been about Hololens, which was never going to be suitable here. But apparently I can't un-mark anything as an answer...
The coming mixed reality headsets are full VR with twin cameras, seemingly exactly what I want? But any chance they are? Thank you.