Voice Input based on 101E not working

SashaFC

Hi, I am trying to add two simple voice commands to my program- "map on" and "map off", to turn on and off the 'draw visual meshes' option within Spatial Mapping (Prefab from Hologram Academy). I followed 101E, and put this Speech Manager script on the Spatial Mapping game object:

using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using UnityEngine.Windows.Speech;

public class SpeechManager : MonoBehaviour
{
KeywordRecognizer keywordRecognizer = null;
Dictionary<string, System.Action> keywords = new Dictionary<string, System.Action>();

// Use this for initialization
void Start()
{
    keywords.Add("Map On", () =>
    {
        // Call the OnReset method on every descendant object.
        this.BroadcastMessage("OnMap");
    });

    keywords.Add("Map Off", () =>
    {
        // Call the OnReset method on every descendant object.
        this.BroadcastMessage("OffMap");
    });


    // Tell the KeywordRecognizer about our keywords.
    keywordRecognizer = new KeywordRecognizer(keywords.Keys.ToArray());

    // Register a callback for the KeywordRecognizer and start recognizing!
    keywordRecognizer.OnPhraseRecognized += KeywordRecognizer_OnPhraseRecognized;
    keywordRecognizer.Start();
}

private void KeywordRecognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
{
    System.Action keywordAction;
    if (keywords.TryGetValue(args.text, out keywordAction))
    {
        keywordAction.Invoke();
    }
}

}

I also added to functions to the existing SpatialMapping class that was included with the original Spatial Mapping prefab:

void OnMap()
{
drawVisualMeshes = true;
}

void OffMap()
{
    drawVisualMeshes = false;
}

My microphone is enabled both in unity and on the hololens' settings. "Hey Cortana" seems to work fine, so I know it isn't an issue with the microphone, but when I deployed my app, the commands had no effect on the scene.

Can anyone tell me what I'm doing wrong?

Thanks

HoloSheep

@SashaFC if you put a breakpoint inside your KeywordRecognizer_OnPhraseRecognized routine on the

keywordAction.Invoke();

line does it break when you say either of the voice commands at runtime?

If not, then double check that your microphone capability is check in the Unity Project Settings | Player | Publish Settings | Capabilities
section

If it does break there, then the voice command is being recognized so you then want to look at your handler and make sure it is getting fired. If it is not, you will want to check your wiring and where you are add your script components.

If the handler is getting fired, but you are not getting the results you want then there is likely an issue with the logic of the code that you are executing for the actual command action.

HTH

HoloSheep

@SashaFC if you put a breakpoint inside your KeywordRecognizer_OnPhraseRecognized routine on the

keywordAction.Invoke();

line does it break when you say either of the voice commands at runtime?

If not, then double check that your microphone capability is check in the Unity Project Settings | Player | Publish Settings | Capabilities
section

If it does break there, then the voice command is being recognized so you then want to look at your handler and make sure it is getting fired. If it is not, you will want to check your wiring and where you are add your script components.

If the handler is getting fired, but you are not getting the results you want then there is likely an issue with the logic of the code that you are executing for the actual command action.

HTH

SashaFC

@HoloSheep thanks for your debugging advice! I put in those debug statements and realized it was code logic issue and was able to fix it. Thank you

DrMax

@HoloSheep I have tried these debugging steps but nothing is catching on the breakpoints in Visual Studio.
The Origami demo works fine with voice, but when i build my own app it deos not run in the emulator, but i can see my voice commands working fine within the Unity Editor.
Have confirmed my mic is set up correctly in player prefrences.
What should i try next?

HoloSheep

@DrMax

When you say:

I have tried these debugging steps but nothing is catching on the breakpoints in Visual Studio.

have you been able to confirm that Visual Studio is breaking correctly in debug mode by putting a break point at some line of code that you know is running (for example in the Start method of a gameobject that does get instantiated in your scene? If not, when you run the app are you using "Start Debugging" (pressing F5)? Have you tried changing your solution configuration from Release to Debug?

If Visual Studio is breaking on some break points but not on the ones associated to your voice commands, then as you suggest it is likely something to do with your setup and/or configuration.

Also when you say:

The Origami demo works fine with voice, but when i build my own app it deos not run in the emulator, but i can see my voice commands working fine within the Unity Editor.

If you own app does not "run" in the emulator, do you mean it just doesn't run or the app runs but the voice commands are not working?

hassank

My team created a tutorial showing a bunch of Hololens features including voice recognition. Check it out! https://youtu.be/FOkbfOJEDDk