Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

MMVM/MVC for HoloLens development

Data-driven apps retrieve data from servers and dynamically create UI element collections. MMVM/MVC patterns simplify UI-heavy data-driven enterprise app development, and I wanted to ask how people are doing this in Unity.

Here are some choices we've used or looked at:

StrangeIoC: open-source MVC and dependency injection framework, results in very modular apps, but has a steep learning curve, and creates a lot of boilerplate (publish/subscribe to events)

NoesisGUI: framework for designing UIs outside of Unity using familiar XAML design tools, follows WPF conventions and APIs, but you only see UIs in Unity Play mode

uFrame: shares best characteristics of PlayMaker (visual coding) and StrangeIoC (MMVM pattern), but you need to buy into their top-down workflow, and understand generated code patterns

ScaleForm: use Flash UI design tools with Unity
CoherentUI: use HTML/JavaScript to create dynamic UIs
MarkUX: simple framework, but uses proprietary XML markup syntax

While StrangeIoC was very powerful, it resulted in too much boilerplate, so we normally roll our own MVC for smaller apps. This is not scalable on large teams where separating designer and developer workflows is beneficial for project maintainability.

Please share your Unity MMVM/MVC development stories here, so we can learn and come up with best practices in this area.

It would be great to see WPF support for non-XAML Unity apps, so 3D apps don't need to exit exclusive mode when showing UIs, and I'd love to hear from the HoloLens team on that as well.

Answers

  • ContextVRContextVR ✭✭✭

    Decoupling designer/developer workflows aside, we've had good success creating views as collections of instantiated prefabs, which were both 3D objects and uGUI Canvas elements. Anyone else wants to chime in?

  • ContextVRContextVR ✭✭✭

    Decoupling designer/developer workflows aside, we've had good success creating views as collections of instantiated prefabs, which were both 3D objects and uGUI Canvas elements. Anyone else wants to chime in?

  • subere23subere23 ✭✭
    edited May 2016

    I am not advanced enough in term of patterns to add anything to the conversation yet, but it is one I hope others do chime in on.

    In terms of UX/UI, I have been experimenting and iterating a ton of various UI styles for hololens. The project I am gearing up to make with have many screens with lots of info, so finding a comfortable way to put that around the user is high priority with ease of use coming a close second. So far I find that obvious to the case of Unity UI to be fast at iterating as well as performing, while a mixture of 3D and U-UI seems to feel the best.

    I think the 3D portions really allow a "pictures are worth 1000 words" abstraction to lighten the amount of text we have to use.

  • ContextVRContextVR ✭✭✭

    Thanks @subere23, similar to VR headsets, HoloLens forces you to design apps without having familiar input modalities, such as keyboard, mouse or touch, so voice guidance and voice recognition could help. It might also be useful to look at some VR design guidelines, such as avoiding head-locked objects, moving the camera, having text/buttons too close or too close, etc.

    Is your project retrieving data dynamically from a cloud-based resource?

  • The project we are digging into is a multi phase project, the first phase is a completely AR/VR single user experience. At phase 2 it will become a multi user/multi platform experience. At that point I do believe we will move to a heavy cloud-based model. We are currently exploring Azure as the solution for that end, which we will likely end up using.

  • ContextVRContextVR ✭✭✭

    Sounds like a plan :) Microsoft just announced Windows Holographic support for other AR/VR devices, so hopefully vendors will start jumping onboard. It would be great to have HoloTookit working on a larger device base, including Vives and Rifts.

  • JasonJason ✭✭

    Develop once deploy everywhere; cross-platform is the way to go. Fundamental question for your effort: how will you make money?

  • Money isn't the chief concern. My HoloTwerk is a labor of love.

  • JasonJason ✭✭

    Ha, sweet, yeah this is incredible technology and easy to be passionate about.

  • ContextVRContextVR ✭✭✭

    I wonder what app prices the market will bear given that current hi-end AR/VR solutions hover around $3K price point.

  • JasonJason ✭✭

    I think most consumers expect free app downloads. Apps could have monetizing methods built into its design (i.e. offer a free demo but with a paid version, monthly recurring auto-renewal subscription services, etc). The app design should have some hooking element to draw the user in and want to spend money. An example of this would be with a program I subscribed for in the past called GuitarBots. I had no problem paying the annual subscription fee of $100 or whatever it was (until the company was sold to Yousician at which point I never renewed my membership due to lack of time). The app initially drew me in with a demo of the software and that created value for me however what interested me was the user driven content producing community.

    The app space will soon become saturated/ competitive by the established players. You could have the best app in the world but unless you pump the marketing dollars into your app, the probability of anyone knowing about or caring about your app reduces over time as the competition increases. You just become another app in the vast ocean of apps.

    My 'ideas' road-map questions whether the idea is niche, scalable, and repeatable (make once sell many) (as examples of my process). Stay away from the 'run-like-hell' strategies especially if there is nothing proprietary about your efforts. Ideally building a large content producing user community is a great goal.

    Excuse my digression from your initial topic however I think it is important to bounce around ideas especially around the question why are you building apps in the first place?

  • @ContextVR Have you checked UniRX's Model-View-(Reactive)Presenter pattern? https://github.com/neuecc/UniRx#ugui-integration
    UniRx is Reactive Extensions for Unity and makes it a lot easier to handle asynchronous inputs.

  • @antao_almada Hi there, on the github page it states "Rx is a library for composing asynchronous and event-based programs using observable collections and LINQ-style query operators".

    After checking the Windows Holographic performance recommendations page for Unity, it states not to use LINQ as it causes heavy allocations affecting CPU performance.

    https://developer.microsoft.com/en-us/windows/holographic/performance_recommendations_for_unity

  • @Jason

    Hi, I'm creator of UniRx, thank you for see the page.

    LINQ causes heavy allocations is true.

    In Unity, "LINQ" means LINQ to Objects.
    Allocations point to Where, Select... so create Enumerable objects on number of method chains.
    And when run called GetEnumerator... so craete Enumerator objects on number of method chains.
    If use in lambda and capture variable... create hidden class by compiler, it is allocation.

    In UniRx?

    Allocations point to Where, Select... so create Observable objects on number of method chains.
    And when run called Subscribe... so craete IDisposable objects on number of method chains.
    If use in lambda and capture variable... create hidden class by compiler, it is allocation.

    Same.

    But the difference is sequence lifetime.
    I avoid use LINQ to Objects on tight loop(for example Update), too.
    UniRx(Reactive Pattern) is mainly create and subscribe is only once, and running long time.
    If runninng in pipeline, does not allocate more(if you use simple method).

    So UniRx does not affect memory allocation and performance.
    Sometimes, there is a case more of UniRx performance is good.
    For example, UniRx's internal MicroCoroutine is faster than Unity's standard coroutine.
    https://github.com/neuecc/UniRx#microcoroutine
    If you write hand coroutine, maybe UniRx is faster than it.
    UniRx is heavily tuned for game use.

    Another about LINQ.
    I've creating LINQ to GameObject.
    https://github.com/neuecc/LINQ-to-GameObject-for-Unity/
    It is LINQ but fast and zero memory allocate(If does not connect to LINQ to Objects)
    In many cases, writing the loop for traverse on their own, slow than LINQ to GameObject.
    (because LINQ to GameObject is heavily optimized by using struct enumerator, hand stack pool, and uses many other optimize techinique...)

    So LINQ is slow and LINQ is heavy allocations is true but not true.

  • JasonJason ✭✭
    edited August 2016

    @neuecc Hi, thanks for the additional information and clarifications. I look forward to checking out your UniRx creation. Have you tried UniRx with any HoloLens apps?

    How would app performance be as HoloLens apps need to maintain 60 fps? I see several use cases with your technology however I am reserved with time as we all are.

  • JasonJason ✭✭
    edited August 2016

    PRODUCT DESIGN

    LINQ put my mind to thinking about data and HoloLens app performance. I'm seeing people use chatbots in their app design.

    https://www.youtube.com/watch?v=8ozfw2LO6No

    Conversation as a platform utilizing gaze, gesture, and voice is pretty significant and a powerful new UI. This is a neat way to go:

    https://www.microsoft.com/cognitive-services/en-us/language-understanding-intelligent-service-luis

    Plug in additional Oxford APIs and you can keep yourself busy I'm sure:

    https://www.microsoft.com/cognitive-services

    The API is fast as well:

    https://dev.projectoxford.ai/docs/services/56ea598f778daf01942505ff/operations/56ea5a1cca73071fd4b102bb/console

    You can have a holographic salesman standing there selling you cars when you swatch the color options.

    https://www.youtube.com/watch?v=rpZvP6Mq1js

    With a hyperlink to a 2D app window checkout solution through PayPal and I think you have eCommerce.

    You might be able to plug in PayPal's REST or NVP/SOAP API endpoints into the chatbot and you are good to go:

    https://developer.paypal.com/docs/api/overview/

    Probably should be a separate thread but this was just me thinking through typing.

    Thoughts?

  • Unity executes in a render-update loop and this has to run as fast as possible. When running asynchronous code, the reply to a request may take several frames to arrive, so you should use coroutines. The handling of multiple asynchronous data sources may become buggy and hard to maintain. This is where Reactive Extensions shines, where everything is a stream and with its solid set of operators: http://reactivex.io/

  • I haven't "announced" this anywhere else before, but I might as well announce it now.

    I'm working on an open source project called Adept. The goal is to build code sharing bridges between the Unity UWP world and the traditional Xaml UWP world with the goal of sharing as much code as possible between the two.

    One of the very first capabilities I created was two-way data binding for Unity. I support well-known interfaces like INotifyPropertyChanged and IObservableCollection. I introduced basic binding constructs into Unity that allow you to bind property values from objects into property values in Unity UI controls. I support the same IValueConverter interface already supported in Xaml and I even support item templates. Just like in Xaml, ItemsControls have an ItemTemplate property and this template gets instantiated for every item in the list. But in Unity the ItemTemplate is a Unity Prefab that can contain Unity UI elements and those UI elements can have bindings. I have prefabs for common Xaml controls like ListBox and ComboBox, etc. I even support completely custom bindings. For example, you may want to load a 3D object dynamically whenever the SelectedModel property changes. This could, for example, be handled by a custom binding to a SelectedModel.Mesh property of the ViewModel.

    This binding layer enables Unity applications to follow the MVVM pattern and even share the same Models and ViewModels used by the Xaml version of the application. I believe this is a huge step forward in code reuse and I believe it could evolve into an invaluable tool for companion applications.

    All of this code is open sourced and you can start using it today. The bad news is it's not currently documented (beyond my rather liberal use of code comments).

    If you want to take a look at the project you can check it out here:

    https://github.com/jbienzms/Adept

    Most of the binding stuff can be found here.

    If you want to see the framework in use can check out this YouTube video here:

    https://youtu.be/MrZd3Ldks14

    And finally, here's a screenshot:

    This work is still quite new and the project will be ongoing. If you think you might be interested in contributing, let me know.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • I haven't "announced" this anywhere else before, but I might as well announce it now.

    I'm working on an open source project called Adept. The goal is to build code sharing bridges between the Unity UWP world and the traditional Xaml UWP world with the goal of sharing as much code as possible between the two.

    One of the very first capabilities I created was two-way data binding for Unity. I support well-known interfaces like INotifyPropertyChanged and IObservableCollection. I introduced basic binding constructs into Unity that allow you to bind property values from objects into property values in Unity UI controls. I support the same IValueConverter interface already supported in Xaml and I even support item templates. Just like in Xaml, ItemsControls have an ItemTemplate property and this template gets instantiated for every item in the list. But in Unity the ItemTemplate is a Unity Prefab that can contain Unity UI elements and those UI elements can have bindings. I have prefabs for common Xaml controls like ListBox and ComboBox, etc. I even support completely custom bindings. For example, you may want to load a 3D object dynamically whenever the SelectedModel property changes. This could, for example, be handled by a custom binding to a SelectedModel.Mesh property of the ViewModel.

    This binding layer enables Unity applications to follow the MVVM pattern and even share the same Models and ViewModels used by the Xaml version of the application. I believe this is a huge step forward in code reuse and I believe it could evolve into an invaluable tool for companion applications.

    All of this code is open sourced and you can start using it today. The bad news is it's not currently documented (beyond my rather liberal use of code comments).

    If you want to take a look at the project you can check it out here:

    https://github.com/jbienzms/Adept

    Most of the binding stuff can be found here.

    If you want to see the framework in use can check out this YouTube video here:

    https://youtu.be/MrZd3Ldks14

    And finally, here's a screenshot:

    This work is still quite new and the project will be ongoing. If you think you might be interested in contributing, let me know.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • I haven't "announced" this anywhere else before, but I might as well announce it now.

    I'm working on an open source project called Adept. The goal is to build code sharing bridges between the Unity UWP world and the traditional Xaml UWP world with the goal of sharing as much code as possible between the two.

    One of the very first capabilities I created was two-way data binding for Unity. I support well-known interfaces like INotifyPropertyChanged and IObservableCollection. I introduced basic binding constructs into Unity that allow you to bind property values from objects into property values in Unity UI controls. I support the same IValueConverter interface already supported in Xaml and I even support item templates. Just like in Xaml, ItemsControls have an ItemTemplate property and this template gets instantiated for every item in the list. But in Unity the ItemTemplate is a Unity Prefab that can contain Unity UI elements and those UI elements can have bindings. I have prefabs for common Xaml controls like ListBox and ComboBox, etc. I even support completely custom bindings. For example, you may want to load a 3D object dynamically whenever the SelectedModel property changes. This could, for example, be handled by a custom binding to a SelectedModel.Mesh property of the ViewModel.

    This binding layer enables Unity applications to follow the MVVM pattern and even share the same Models and ViewModels used by the Xaml version of the application. I believe this is a huge step forward in code reuse and I believe it could evolve into an invaluable tool for companion applications.

    All of this code is open sourced and you can start using it today. The bad news is it's not currently documented (beyond my rather liberal use of code comments).

    If you want to take a look at the project you can check it out here:

    https://github.com/jbienzms/Adept

    Most of the binding stuff can be found here.

    If you want to see the framework in use can check out this YouTube video here:

    https://youtu.be/MrZd3Ldks14

    And finally, here's a screenshot:

    This work is still quite new and the project will be ongoing. If you think you might be interested in contributing, let me know.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • I haven't "announced" this anywhere else before, but I might as well announce it now.

    I'm working on an open source project called Adept. The goal is to build code sharing bridges between the Unity UWP world and the traditional Xaml UWP world with the goal of sharing as much code as possible between the two.

    One of the very first capabilities I created was two-way data binding for Unity. I support well-known interfaces like INotifyPropertyChanged and IObservableCollection. I introduced basic binding constructs into Unity that allow you to bind property values from objects into property values in Unity UI controls. I support the same IValueConverter interface already supported in Xaml and I even support item templates. Just like in Xaml, ItemsControls have an ItemTemplate property and this template gets instantiated for every item in the list. But in Unity the ItemTemplate is a Unity Prefab that can contain Unity UI elements and those UI elements can have bindings. I have prefabs for common Xaml controls like ListBox and ComboBox, etc. I even support completely custom bindings. For example, you may want to load a 3D object dynamically whenever the SelectedModel property changes. This could, for example, be handled by a custom binding to a SelectedModel.Mesh property of the ViewModel.

    This binding layer enables Unity applications to follow the MVVM pattern and even share the same Models and ViewModels used by the Xaml version of the application. I believe this is a huge step forward in code reuse and I believe it could evolve into an invaluable tool for companion applications.

    All of this code is open sourced and you can start using it today. The bad news is it's not currently documented (beyond my rather liberal use of code comments).

    If you want to take a look at the project you can check it out here:

    https://github.com/jbienzms/Adept

    Most of the binding stuff can be found here.

    If you want to see the framework in use can check out this YouTube video here:

    https://youtu.be/MrZd3Ldks14

    And finally, here's a screenshot:

    This work is still quite new and the project will be ongoing. If you think you might be interested in contributing, let me know.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • I haven't "announced" this anywhere else before, but I might as well announce it now.

    I'm working on an open source project called Adept. The goal is to build code sharing bridges between the Unity UWP world and the traditional Xaml UWP world with the goal of sharing as much code as possible between the two.

    One of the very first capabilities I created was two-way data binding for Unity. I support well-known interfaces like INotifyPropertyChanged and IObservableCollection. I introduced basic binding constructs into Unity that allow you to bind property values from objects into property values in Unity UI controls. I support the same IValueConverter interface already supported in Xaml and I even support item templates. Just like in Xaml, ItemsControls have an ItemTemplate property and this template gets instantiated for every item in the list. But in Unity the ItemTemplate is a Unity Prefab that can contain Unity UI elements and those UI elements can have bindings. I have prefabs for common Xaml controls like ListBox and ComboBox, etc. I even support completely custom bindings. For example, you may want to load a 3D object dynamically whenever the SelectedModel property changes. This could, for example, be handled by a custom binding to a SelectedModel.Mesh property of the ViewModel.

    This binding layer enables Unity applications to follow the MVVM pattern and even share the same Models and ViewModels used by the Xaml version of the application. I believe this is a huge step forward in code reuse and I believe it could evolve into an invaluable tool for companion applications.

    All of this code is open sourced and you can start using it today. The bad news is it's not currently documented (beyond my rather liberal use of code comments).

    If you want to take a look at the project you can check it out here:

    https://github.com/jbienzms/Adept

    Most of the binding stuff can be found here.

    If you want to see the framework in use can check out this YouTube video here:

    https://youtu.be/MrZd3Ldks14

    And finally, here's a screenshot:

    This work is still quite new and the project will be ongoing. If you think you might be interested in contributing, let me know.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • It looks like there are some quality existing MVC frameworks for Unity, such as MarkLight (http://www.marklightforunity.com/). However I too am looking to use MVC/MVVM in C++ Holographic apps, and am trying to wrap my mind around the best way to do that.

    The Windows-universal-samples holographic examples are great, but the code is completely separated for the computer (what the code classes and drawing technique is, i.e. texture, shader, asset-loader) rather than for humans (separation of concerns, i.e. data manipulation, user interface, user interaction controller). The MVC pattern would make it way easier to navigate a new project.

    I think the first thing to tackle, would be some kind of observable data store. Then next a good way to get components to "render themselves" (that is, reactive-ly) rather than having to be controlled by global state.

    Also @jbienzms it seems there are several copies of your comment in this thread :).

  • UGH. Thanks for pointing that out @doublerebel. I would post it and then it would disappear. It turns out it was being held for moderator approval. Probably because of the YouTube video links or something. Unfortunately it doesn't look like the system will let me delete those duplicate posts. Sorry about that.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • @neuecc said:
    So LINQ is slow and LINQ is heavy allocations is true but not true.

    So which is it?

    Stephen Hodgson
    Microsoft HoloLens Agency Readiness Program
    Virtual Solutions Developer at Saab
    HoloToolkit-Unity Moderator

  • @jbienzms I am trying to use your data binding approach in my Hololens app. I have few questions.

    1. I do not see any commits to the code for last 3 months. Is it still an active project which will be improved?
    2. Does this also support the view model to fetch data from my Azure REST API? What is the extension point for that?
  • @Sreeni Thanks for asking. This is still an active passion but not unfortunately not much of an active project. I have a personal interest in investing the time into it but my current workload has me allocated to solving other problems. Part of the reason I haven't been able to justify spending time on it is because I haven't found enough projects that need it yet. Up to this point most MVVM projects are existing WPF / UWP 2D apps and most HoloLens apps are Unity. The two worlds haven't attempted to share much code yet (other than my own hobby projects). I expect this to change over the next year and I hope to make a case to spend more time building this bridge. It would be an incredible help if I could document a handful of projects that could benefit from this bridge. It would help make a case for spending more time on it.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • @jbienzms Thanks for publishing this! It looks very interesting, I'm currently developing a data viewing app for HoloLens and for mobile devices. One of the challenges we've run into is how to create a low maintenance code base for interacting with data on HoloLens and from other devices, and this looks like it could do the trick. Will give it a spin and see if I can integrate it into our project.

    Owner of the startup Information Experience Sweden AB, http://informationexperience.se
    Information Management, SharePoint, CAD, Visualization for AEC. Architecting and developing solutions for HoloLens, combined with AR Toolkit, Kinect, RealSense, Leap Motion.

Sign In or Register to comment.