Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

KeywordManager - UnityEvents vs SendMessage

The KeywordManager in the HoloToolkit uses UnityEvents as the way of triggering the Response routines on other game object's component scripts which is different from the SpeechManager sample in the Origami tutorial and from many of the other components in the HoloTolokit that make use of the SendMessage approach.

The Inspector support of the KeywordManager is definitely a plus.
However, I am running into what may be a negative, or maybe I am just missing a simple way to wire things up.

In the app I am working on I am using prefabs for my main gameObject type and those prefabs have script components.

My scene starts with one of the prefabs in the Hierarchy, and if the user chooses to add additional instances of that gameobject at runtime they can do so by a voice command to: "Add _____" which is handled by one of my global manager scripts and that works just fine.

The application also uses the KeywordManager to trigger response routines on both the script components of the initial prefab in the Hierarchy as well as the script components of the cloned prefabs that get instantiated at runtime. And this is where the problem seems to lie.

It took me a fair while to tract down what was happening, but once I did I discovered that the only instance of the scripts that seem to be listening to the Keyword manager's events is the initial prefab that was in the Hierarchy at design time. The ones that get instantiated later at runtime do not have listeners setup and therefore do not receive events from the keyword manager.

Is there a simple way (that I am overlooking) of wiring up the runtime components to also listen for the Keyword manger events?

Windows Holographic User Group Redmond

WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
WinHUGR YouTube Channel -- live streamed meetings

Tagged:

Best Answers

  • ContextVRContextVR ✭✭✭
    edited June 2016 Accepted Answer

    @HoloSheep Sounds like you manually created a prefab instance in the Editor hierarchy and registered a response in KeywordManager that references that prefab instance. When you create additional instances at runtime, they do not have responses registered in KeywordManager.

    I see several ways of addressing this.

    1) Create an ObjectManager in hierarchy that would have keyword responses registered in KeywordManager. ObjectManager would manage prefab instances and dispatch keyword events to the right instance using GazeManager.FocusedObject.

    2) Create an ObjectManager in hierarchy that would create prefab instances and register them in KeywordManager responses array. This is not as clean as 1) but could be useful if you want to handle keywords specific to individual objects.

    3) Create an ObjectManager that would have keyword responses registered in KeyboardManager, and BroadcastMessage to all objects in hierarchy instead of SendMessage to specific instances. In a sense, this would convert the strong binding in KeywordManager to a weak binding through a named Unity event, and the right objects will pick up the event.

    I see number 1 as the best way of doing this.

    a) Decouples KeywordManager from prefab instance lifecycle.
    b) Enables keyword handling specific to object families.
    c) No extra overhead with every added prefab instance.

    Number 3 could be a quick and dirty way of doing this if you want to make it work with your existing way of creating prefab instances.

    Hope this helps.

Answers

  • ContextVRContextVR ✭✭✭
    edited June 2016 Accepted Answer

    @HoloSheep Sounds like you manually created a prefab instance in the Editor hierarchy and registered a response in KeywordManager that references that prefab instance. When you create additional instances at runtime, they do not have responses registered in KeywordManager.

    I see several ways of addressing this.

    1) Create an ObjectManager in hierarchy that would have keyword responses registered in KeywordManager. ObjectManager would manage prefab instances and dispatch keyword events to the right instance using GazeManager.FocusedObject.

    2) Create an ObjectManager in hierarchy that would create prefab instances and register them in KeywordManager responses array. This is not as clean as 1) but could be useful if you want to handle keywords specific to individual objects.

    3) Create an ObjectManager that would have keyword responses registered in KeyboardManager, and BroadcastMessage to all objects in hierarchy instead of SendMessage to specific instances. In a sense, this would convert the strong binding in KeywordManager to a weak binding through a named Unity event, and the right objects will pick up the event.

    I see number 1 as the best way of doing this.

    a) Decouples KeywordManager from prefab instance lifecycle.
    b) Enables keyword handling specific to object families.
    c) No extra overhead with every added prefab instance.

    Number 3 could be a quick and dirty way of doing this if you want to make it work with your existing way of creating prefab instances.

    Hope this helps.

  • ContextVRContextVR ✭✭✭

    @HoloSheep I am glad this helped! Looks like your single instance component is equivalent to the ObjectManager idea.

    Here's probably the simplest way of doing this, which is similar to number 3, but doesn't require any code:

    http://i.imgur.com/DH70u6X.png

    Regarding Prefabs, they are central to efficiently working in Unity and sharing assets across multiple scenes.

    "Unity way" is probably trying simplest things first, so there are multiple ways to get around Prefab limitations while keeping object lifecycle decoupled from keyword recognition.

    Similar to the astronaut example, one can label object-specific keywords in a popup menu that shows when you hover over an object.

    KeywordManager can also be extended to send events to GazeManager.FocusedObject.

  • For the record, as I mentioned above, in the app I am working on I choose not to associate the target instance with gaze and instead to implement the idea of "user selection" since in several of our scenarios we did not want to force the user to constantly gaze at the Hologram instance in order to execute a keyword command.

    So my point is, I don't think the correct solution to making the KeywordManager more flexible in it relationship with prefabs is to tie its logic to the Gaze manager and the concept of gaze.

    If instead the Keyword manager simply allowed you to associate an Object and a method name for a response and then used the SendMessage syntax instead of the UnityEvent syntax , the instantiated object(s) would receive the message even if they were instantiated at runtime and the "This" keyword would be your friend.

    As I pointed out the SpeechManager sample in the Origami project used SendMessage, but the much IDE friendlier version that made it into the HoloToolkit choose the event model path instead.

    Depending on the particular scenario or use case then either all of the objects of that prefab type could respond to the command or individual instance could respond depending on conditional logic like if they were user selected, or if they are the GazeManager.FocusedObject.

    As I understand it object targeted SendMessage is a little less performant that firing an event (which is perhaps why KeywordManager implements events) but it seems like a reasonable tradeoff for voice commands if that's what it takes to get the command to the runtime instantiated versions of prefab objects.

    Broadcasting the message app wide to all objects however seems like too high of a price to pay in anything but the very simplest of projects.

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • ContextVRContextVR ✭✭✭

    The changes proposed in the pull request should make it easy to create different policies for selecting target object to send messages to.

  • @CotextVR, please explain.

    How will changes to the GazeManager help prefab instances receive keywordManager responses in those cases like the ones I have mentioned above where and when gaze is not the relevant or choice modality?

    Did the PR end up including any changes to the KeywordManager that I am overlooking?

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • ContextVRContextVR ✭✭✭

    The PR provides a couple of examples on how to create MessageSender classes and register them in KeywordManager. One is based on object focused on with gaze, another one is based on object explicitly selected with gaze + voice command. Other selection mechanisms are entirely possible, what's the way you'd like to select objects?

  • ContextVRContextVR ✭✭✭

    @HoloSheep The PR has been approved, check it out when you get a chance: https://github.com/Microsoft/HoloToolkit-Unity/pull/72

    It would be great to think about other forms of object selection that would be useful to the community.

Sign In or Register to comment.