Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Can Mixed Reality show you in a real car driving in a VR world?

It is for a project I am working on. If I can avoid a green screen solution that would be good, but first I need to get the prototype working with "something", then can do refinements.

Ideally I want to have a dual camera feed showing a 3d image of the car the person is in and out the windows / mirrors be the conventional "need for speed underground" or similar VR world. Will this tech do it? Any particular pointers on how?

Sure appreciate any help. This is not an "idle" question, you can see what I have done so far at www.vrdriversim.com/moki being the prototype page, and lots of related info on the various tabs and pages there... Thank you.

Best Answers

  • VinceSVinceS
    Accepted Answer

    Thank you for that useful insight. It is key to get visual and physical totally in sync, thus getting motion sickness outcomes to the low end. First is to match physical outcomes to control inputs, and it is a really solid body of work to do to get precise control of the simulator so what is input via the controls matches the sensation. THEN need to get a 100% match to associated VR feed, which I am exploring the smartest ways to do that. For physical realism, I will be datalogging a bunch of real world driving scenarios and matching that precisely on the simulator purely by steering, braking and accelerating, with road noise and bumps coming through in a realistic way.

    For most, this would be pretty tricky work, but I have a really solid methodology to do it - we'll see how we go with that one, but high confidence. Nevertheless, minimum 400 hours work, and I notoriously underestimate tasks! From there I can start answering my own questions. I have just discovered Leap controllers and have one of them on the way as hand tracking from that combined with a vehicle interior overlay and a couple of artificial arms might be enough to get it fully believable. Where do I get a Mazda 2 overlay? Issue for another day!

    When I get there, I will be able to figure out which bits really matter. It is bound to be different from what I am currently thinking! I just data entered all the costs associated with this thing and it adds up to $33k, about double what I was expecting and pretty well maxed me out, for now anyway. There is going to be a month of not much progress as my 57yo business partner of 22 years recently passed away, after a short illness, just as we were about to totally re-pitch the business, so this falls to me and is "a bit of a distraction".

    My current solution to the OP q is to use green screening on the simulator - so painted windows and curtain behind. Not sure about the external mirrors, maybe totally VR generated. Had intended to use the Vive headset camera to show internals of the cab but it may be inadequate. So a couple of mobile ph cameras (you can get stand-alones) stuck on the headset in eye positions and can use Rift or Vive. All the joy of prototyping. But a lot of what I don't know is known, and accessing that is all I am hoping to do while I still have thinking time around this - ie before the self-imposed crisis arrives. Most likely, AR as it is currently being conceived is not the nature of the answer to the application. But love to hear any specific suggestions that could get me closer to real answers. Thank you for that.

Answers

  • VinceSVinceS
    Accepted Answer

    Thank you for that useful insight. It is key to get visual and physical totally in sync, thus getting motion sickness outcomes to the low end. First is to match physical outcomes to control inputs, and it is a really solid body of work to do to get precise control of the simulator so what is input via the controls matches the sensation. THEN need to get a 100% match to associated VR feed, which I am exploring the smartest ways to do that. For physical realism, I will be datalogging a bunch of real world driving scenarios and matching that precisely on the simulator purely by steering, braking and accelerating, with road noise and bumps coming through in a realistic way.

    For most, this would be pretty tricky work, but I have a really solid methodology to do it - we'll see how we go with that one, but high confidence. Nevertheless, minimum 400 hours work, and I notoriously underestimate tasks! From there I can start answering my own questions. I have just discovered Leap controllers and have one of them on the way as hand tracking from that combined with a vehicle interior overlay and a couple of artificial arms might be enough to get it fully believable. Where do I get a Mazda 2 overlay? Issue for another day!

    When I get there, I will be able to figure out which bits really matter. It is bound to be different from what I am currently thinking! I just data entered all the costs associated with this thing and it adds up to $33k, about double what I was expecting and pretty well maxed me out, for now anyway. There is going to be a month of not much progress as my 57yo business partner of 22 years recently passed away, after a short illness, just as we were about to totally re-pitch the business, so this falls to me and is "a bit of a distraction".

    My current solution to the OP q is to use green screening on the simulator - so painted windows and curtain behind. Not sure about the external mirrors, maybe totally VR generated. Had intended to use the Vive headset camera to show internals of the cab but it may be inadequate. So a couple of mobile ph cameras (you can get stand-alones) stuck on the headset in eye positions and can use Rift or Vive. All the joy of prototyping. But a lot of what I don't know is known, and accessing that is all I am hoping to do while I still have thinking time around this - ie before the self-imposed crisis arrives. Most likely, AR as it is currently being conceived is not the nature of the answer to the application. But love to hear any specific suggestions that could get me closer to real answers. Thank you for that.

  • Cr*p, just realised the REASON I asked this Q here in the first place was about the ACER Mixed reality headsets, eg https://www.cnet.com/au/products/acer-windows-mixed-reality-headset/preview/ to see if they were applicable. The discussion has been about Hololens, which was never going to be suitable here. But apparently I can't un-mark anything as an answer...

    The coming mixed reality headsets are full VR with twin cameras, seemingly exactly what I want? But any chance they are? Thank you.

Sign In or Register to comment.