Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Projecting a visible Ray from the motion controllers similar to the default cliffhouse space?

EvoxVREvoxVR
edited March 2018 in Q&A and Discussions

Hi everyone I am working on incorporating the visible controller models into our project and am using the HoloToolkit in unity to do it. I've managed to omit some unnecessary libraries and Hololens features ( such as spatial mapping) to get our build to actually export from unity and everyting runs well in the editor.

For some reason whenever I tweak the LeftHandInputControl and RightHandInputControl prefabs and enable them to display the raycast nothing shows up . When I press a button a cursor dot shows up where I believe a ray would end , unfortunately though the actual line being cast from the controller to this endpoint isn't visible and I'm not quite sure if it's user error on my end or if I'm just not grasping how the holotoolkit works in regards to the motion controllers. Here is an example pic of my layout currently in editor, you can see the model and the cursor / endpoint for the raycast hovering just at the head of the seat but as shown no Connecting Ray between the controller and the endpoint . Does anyone have any advice or resources on how to accomplish something like the default raycasts you can see in the main mixed reality lobby ? Thank you very much !

Best Answer

  • EvoxVREvoxVR
    edited March 2018 Accepted Answer

    @mark_grossnickle thank you So much your code worked , I had to make a few modifications on my end ( like the big ugly If else statements for controller handedness because I was getting an error on an unassigned controller reference declaring it the way you did ) but otherwise the Rays are working great.

    For anyone else looking to do something similar I updated the UpdateControllerState() function in the MotionControllerVisualizer.cs class

    I declared the following variables at the top of the class and Created two LineRenderer Game Objects that correspond to the right and left hand rays and currently exist in the scene .

    //*** Line objects
            public LineRenderer LeftPointerLine ;
            public LineRenderer RightPointerLine;
    
            //*** Line Distance 
            public float lineDistance = 100f;
    
            public bool displayPointerRay;
    
    
    
    private void UpdateControllerState()
            {
    #if UNITY_WSA && UNITY_2017_2_OR_NEWER
    
                foreach (var sourceState in InteractionManager.GetCurrentReading())
                {
                    LineRenderer line;
                    MotionControllerInfo currentController;
    
    
    
    
                    if (sourceState.source.kind == InteractionSourceKind.Controller && controllerDictionary.TryGetValue(GenerateKey(sourceState.source), out currentController))
                    {
    
    
                        if (AnimateControllerModel)
                        {
                            currentController.AnimateSelect(sourceState.selectPressedAmount);
    
                            if (sourceState.source.supportsGrasp)
                            {
                                currentController.AnimateGrasp(sourceState.grasped);
                            }
    
                            if (sourceState.source.supportsMenu)
                            {
                                currentController.AnimateMenu(sourceState.menuPressed);
                            }
    
                            if (sourceState.source.supportsThumbstick)
                            {
                                currentController.AnimateThumbstick(sourceState.thumbstickPressed, sourceState.thumbstickPosition);
                            }
    
                            if (sourceState.source.supportsTouchpad)
                            {
                                currentController.AnimateTouchpad(sourceState.touchpadPressed, sourceState.touchpadTouched, sourceState.touchpadPosition);
                            }
                        }
    
                        Vector3 newPosition;
                        if (sourceState.sourcePose.TryGetPosition(out newPosition, InteractionSourceNode.Grip) && ValidPosition(newPosition))
                        {
                            currentController.ControllerParent.transform.localPosition = newPosition;
    
                            //***  Custom Ray logic
    
                            //InteractionInputSource currentController;
                            //line = currentController == InteractionSourceHandedness.Left ? LeftPointerLine : RightPointerLine;
    
                            if(currentController.Handedness == InteractionSourceHandedness.Left){
                                Debug.LogWarning("Left Controller Active, creating Ray !!!!");
                                line = LeftPointerLine;
    
                                if(displayPointerRay && line != null){
    
                                    //***  Custom Ray Logic
                                    if(!line.enabled) 
                                        line.enabled = true;
    
                                    Vector3[] linePositions = new Vector3[2];
                                    linePositions[0] = currentController.ControllerParent.transform.position;
    
                                    // // Original writers cursore snippet var inputSourcePointer ;   [=  TQCursorManager.instance.pointer as InputSourcePointer]
    
                                    Quaternion pointerRotation;
                                    if(sourceState.sourcePose.TryGetRotation(out pointerRotation , InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                    {
                                        linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * lineDistance);
                                    }
    
                                    line.SetPositions(linePositions);
    
    
                                }else{
    
                                    if(line != null && line.enabled)
                                        line.enabled = false;
    
                                }
                            }
    
    
                            if(currentController.Handedness == InteractionSourceHandedness.Right){
                                Debug.LogWarning("Right Controller Active, creating Ray !!!!");
                                line = RightPointerLine;
    
                                if(displayPointerRay && line != null){
    
                                    //*** JJ Custom Ray Logic
                                    if(!line.enabled) 
                                        line.enabled = true;
    
                                    Vector3[] linePositions = new Vector3[2];
                                    linePositions[0] = currentController.ControllerParent.transform.position;
    
                                    // // Original writers cursore snippet var inputSourcePointer ;   [=  TQCursorManager.instance.pointer as InputSourcePointer]
    
                                    Quaternion pointerRotation;
                                    if(sourceState.sourcePose.TryGetRotation(out pointerRotation , InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                    {
                                        linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * lineDistance);
                                    }
    
                                    line.SetPositions(linePositions);
    
    
                                }else{
    
                                    if(line != null && line.enabled)
                                        line.enabled = false;
    
                                }
    
                            }
    
                            //*** end custom Ray logic 
                        }
    
    
                        Quaternion newRotation;
                        if (sourceState.sourcePose.TryGetRotation(out newRotation, InteractionSourceNode.Grip) && ValidRotation(newRotation))
                        {
                            currentController.ControllerParent.transform.localRotation = newRotation;
                        }
    
                        //DebugInteractionSourceState source;
    
                        //source = (DebugInteractionSourceState) sourceState;
    
                        /*Ray newRay;
                        if (sourceState.SourcePose.TryGetPointerRay(out newRay))
                        {
                            if (ShowPointingRay && Physics.Raycast(newRay))
                            {
                                Debug.Log ("!!!!!!!!DRAWING RAY!!!!!!!!!!");
                                // TODO shanama: get pretty ray here, maybe an "active" ray and an "inactive" ray for when buttons are pressed
                                Debug.DrawRay(newRay.origin, newRay.direction, Color.cyan);
                                DrawLine (newRay.origin, newRay.direction, Color.white);
                            }
                        }*/
                    }
                }
    #endif
            }
    

    Here's a picture of my end result sorry it's not better taking a screenshot in editor while holding the controller and headset at an angle to view everything was tricky :

Answers

  • mark_grossnicklemark_grossnickle ✭✭✭
    edited March 2018

    How are you tweaking the prefab? I may not be on the latest toolkit so perhaps they added something but last I checked there was no option for adding a raycast line.

    We ended up adding our own. We added a LineRenderer for each hand (LeftPointerLine and RightPointerLine).

    Within UpdateControllerState's foreach loop of states:
    line = currentController.Handedness == InteractionSourceHandedness.Left ? LeftPointerLine : RightPointerLine;

    Then when it goes to set the controller's position update the lines here. We use a custom CursorManager so you will need to change that to your cursor's position (commented line below):

    `
    {
    Vector3 newPosition;
    if (sourceState.sourcePose.TryGetPosition(out newPosition, InteractionSourceNode.Grip) && ValidPosition(newPosition))
    {
    currentController.ControllerParent.transform.localPosition = newPosition;
    if (DisplayPointerRay && line != null)
    {

                            if (!line.enabled) line.enabled = true;
    
                            Vector3[] linePositions = new Vector3[2];
                            linePositions[0] = currentController.ControllerParent.transform.position;
                            var inputSourcePointer = TQCursorManager.Instance.Pointer as InputSourcePointer;
    
                            if (inputSourcePointer!=null && inputSourcePointer.InputSourceId == sourceState.source.id) //GazeManager.Instance !=  TQCursorManager.Instance.Pointer as GazeManager)
                            { 
                                linePositions[1] = TQCursorManager.Instance.CursorsTransform.position; // Note you will need to change this
                            } else
                            { 
                                Quaternion pointerRotation;
                                if (sourceState.sourcePose.TryGetRotation(out pointerRotation, InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                {
                                    linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * LinePointerDistance);
                                }
                            }
    
    
                            line.SetPositions(linePositions);
                        }  
                    } else
                    {
                        if (line != null && line.enabled) line.enabled = false;
                    }
    

    }`

    Taqtile

  • @mark_grossnickle said:
    How are you tweaking the prefab? I may not be on the latest toolkit so perhaps they added something but last I checked there was no option for adding a raycast line.

    We ended up adding our own. We added a LineRenderer for each hand (LeftPointerLine and RightPointerLine).

    Within UpdateControllerState's foreach loop of states:
    line = currentController.Handedness == InteractionSourceHandedness.Left ? LeftPointerLine : RightPointerLine;

    Then when it goes to set the controller's position update the lines here. We use a custom CursorManager so you will need to change that to your cursor's position (commented line below):

    `
    {
    Vector3 newPosition;
    if (sourceState.sourcePose.TryGetPosition(out newPosition, InteractionSourceNode.Grip) && ValidPosition(newPosition))
    {
    currentController.ControllerParent.transform.localPosition = newPosition;
    if (DisplayPointerRay && line != null)
    {

                            if (!line.enabled) line.enabled = true;
                            
                            Vector3[] linePositions = new Vector3[2];
                            linePositions[0] = currentController.ControllerParent.transform.position;
                            var inputSourcePointer = TQCursorManager.Instance.Pointer as InputSourcePointer;
    
                            if (inputSourcePointer!=null && inputSourcePointer.InputSourceId == sourceState.source.id) //GazeManager.Instance !=  TQCursorManager.Instance.Pointer as GazeManager)
                            { 
                                linePositions[1] = TQCursorManager.Instance.CursorsTransform.position; // Note you will need to change this
                            } else
                            { 
                                Quaternion pointerRotation;
                                if (sourceState.sourcePose.TryGetRotation(out pointerRotation, InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                {
                                    linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * LinePointerDistance);
                                }
                            }
    
    
                            line.SetPositions(linePositions);
                        }  
                    } else
                    {
                        if (line != null && line.enabled) line.enabled = false;
                    }
    

    }`

    Awesome Mark I will replicate this in my project and report back , I appreciate the help !

  • FYI, this is also coming in the Mixed Reality Toolkit soonish

  • @mark_grossnickle In response to how I am modifying the prefabs I'm just enabling and disabling the various booleans for showing rays , and rendering lines. They work for the debug rays but no Rays actually visible in the game view.

  • Taqtile

  • EvoxVREvoxVR
    edited March 2018 Accepted Answer

    @mark_grossnickle thank you So much your code worked , I had to make a few modifications on my end ( like the big ugly If else statements for controller handedness because I was getting an error on an unassigned controller reference declaring it the way you did ) but otherwise the Rays are working great.

    For anyone else looking to do something similar I updated the UpdateControllerState() function in the MotionControllerVisualizer.cs class

    I declared the following variables at the top of the class and Created two LineRenderer Game Objects that correspond to the right and left hand rays and currently exist in the scene .

    //*** Line objects
            public LineRenderer LeftPointerLine ;
            public LineRenderer RightPointerLine;
    
            //*** Line Distance 
            public float lineDistance = 100f;
    
            public bool displayPointerRay;
    
    
    
    private void UpdateControllerState()
            {
    #if UNITY_WSA && UNITY_2017_2_OR_NEWER
    
                foreach (var sourceState in InteractionManager.GetCurrentReading())
                {
                    LineRenderer line;
                    MotionControllerInfo currentController;
    
    
    
    
                    if (sourceState.source.kind == InteractionSourceKind.Controller && controllerDictionary.TryGetValue(GenerateKey(sourceState.source), out currentController))
                    {
    
    
                        if (AnimateControllerModel)
                        {
                            currentController.AnimateSelect(sourceState.selectPressedAmount);
    
                            if (sourceState.source.supportsGrasp)
                            {
                                currentController.AnimateGrasp(sourceState.grasped);
                            }
    
                            if (sourceState.source.supportsMenu)
                            {
                                currentController.AnimateMenu(sourceState.menuPressed);
                            }
    
                            if (sourceState.source.supportsThumbstick)
                            {
                                currentController.AnimateThumbstick(sourceState.thumbstickPressed, sourceState.thumbstickPosition);
                            }
    
                            if (sourceState.source.supportsTouchpad)
                            {
                                currentController.AnimateTouchpad(sourceState.touchpadPressed, sourceState.touchpadTouched, sourceState.touchpadPosition);
                            }
                        }
    
                        Vector3 newPosition;
                        if (sourceState.sourcePose.TryGetPosition(out newPosition, InteractionSourceNode.Grip) && ValidPosition(newPosition))
                        {
                            currentController.ControllerParent.transform.localPosition = newPosition;
    
                            //***  Custom Ray logic
    
                            //InteractionInputSource currentController;
                            //line = currentController == InteractionSourceHandedness.Left ? LeftPointerLine : RightPointerLine;
    
                            if(currentController.Handedness == InteractionSourceHandedness.Left){
                                Debug.LogWarning("Left Controller Active, creating Ray !!!!");
                                line = LeftPointerLine;
    
                                if(displayPointerRay && line != null){
    
                                    //***  Custom Ray Logic
                                    if(!line.enabled) 
                                        line.enabled = true;
    
                                    Vector3[] linePositions = new Vector3[2];
                                    linePositions[0] = currentController.ControllerParent.transform.position;
    
                                    // // Original writers cursore snippet var inputSourcePointer ;   [=  TQCursorManager.instance.pointer as InputSourcePointer]
    
                                    Quaternion pointerRotation;
                                    if(sourceState.sourcePose.TryGetRotation(out pointerRotation , InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                    {
                                        linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * lineDistance);
                                    }
    
                                    line.SetPositions(linePositions);
    
    
                                }else{
    
                                    if(line != null && line.enabled)
                                        line.enabled = false;
    
                                }
                            }
    
    
                            if(currentController.Handedness == InteractionSourceHandedness.Right){
                                Debug.LogWarning("Right Controller Active, creating Ray !!!!");
                                line = RightPointerLine;
    
                                if(displayPointerRay && line != null){
    
                                    //*** JJ Custom Ray Logic
                                    if(!line.enabled) 
                                        line.enabled = true;
    
                                    Vector3[] linePositions = new Vector3[2];
                                    linePositions[0] = currentController.ControllerParent.transform.position;
    
                                    // // Original writers cursore snippet var inputSourcePointer ;   [=  TQCursorManager.instance.pointer as InputSourcePointer]
    
                                    Quaternion pointerRotation;
                                    if(sourceState.sourcePose.TryGetRotation(out pointerRotation , InteractionSourceNode.Pointer) && ValidRotation(pointerRotation))
                                    {
                                        linePositions[1] = sourceState.headPose.position + (pointerRotation * Vector3.forward * lineDistance);
                                    }
    
                                    line.SetPositions(linePositions);
    
    
                                }else{
    
                                    if(line != null && line.enabled)
                                        line.enabled = false;
    
                                }
    
                            }
    
                            //*** end custom Ray logic 
                        }
    
    
                        Quaternion newRotation;
                        if (sourceState.sourcePose.TryGetRotation(out newRotation, InteractionSourceNode.Grip) && ValidRotation(newRotation))
                        {
                            currentController.ControllerParent.transform.localRotation = newRotation;
                        }
    
                        //DebugInteractionSourceState source;
    
                        //source = (DebugInteractionSourceState) sourceState;
    
                        /*Ray newRay;
                        if (sourceState.SourcePose.TryGetPointerRay(out newRay))
                        {
                            if (ShowPointingRay && Physics.Raycast(newRay))
                            {
                                Debug.Log ("!!!!!!!!DRAWING RAY!!!!!!!!!!");
                                // TODO shanama: get pretty ray here, maybe an "active" ray and an "inactive" ray for when buttons are pressed
                                Debug.DrawRay(newRay.origin, newRay.direction, Color.cyan);
                                DrawLine (newRay.origin, newRay.direction, Color.white);
                            }
                        }*/
                    }
                }
    #endif
            }
    

    Here's a picture of my end result sorry it's not better taking a screenshot in editor while holding the controller and headset at an angle to view everything was tricky :

Sign In or Register to comment.