How to develop 2D+3D hybrid apps (C#)?

Palma · August 2016

Hello Nekronomiv,

I tried to follow your tutorial but it's quite hard with so little details.

I did my best to follow your instructions but when I do the build, I get a lot of "Type or namespace name could not be found"

Think you could share that blank app with the modifications?

Also, will this allow me to open my already existing Unity Scene from the XAML? I do not know if there's a difference between the DX Holographic app and the way most of us mortals are doing of Unity>Build to VS>Hololens

Thanks in advance

Edit *

I have been browsing throughout the different topics and I realize that my current Unity project is not DirectX. I have a normal Unity project in which I need to swap to a XAML one because Unity has limitations in terms of code and I have a lot of other things done already in XAML.

So in short, if anyone can give me a clear tutorial on how do I swap from my 3D app (not DirectX) to a 2D one I'd greatly appreciate that. I do not even need to pass any variables. Of course it'd help.
And of course if the only option is DirectX I will go study how I can teleport my current scenes and so on to DirectX.

jeffgonzaga · February 2017

Does anyone know how the stock 'Holograms' UWP app work in HoloLens. I am trying the create an app that should behave similarly to that but it's difficult to find any reference.

thebanjomatic · February 2017

@jeffgonzaga The short answer is that you can't at the moment. The Holograms and Photos apps are created using internal API's that we don't have access to in the public SDK. Hopefully this kind of functionality is in the pipeline, but there hasn't been much said about it officially.

This feature request being "under consideration" excites me though:

https://developer.microsoft.com/en-us/windows/platform/features/enableuwpswithdepthandvolumetolookmorerealisticin3dhmdenvironments/

Of course it would be more exciting if it was "In development" instead

jeffgonzaga · February 2017

@thebanjomatic Thanks for the info. Looks like I have no choice but the do full on Unity dev on this. Similarly, embedding HTML web viewer is Unity has the same challenge

jeffgonzaga · February 2017

> @jbienzms said:
> There are several options in the Unity asset store for embedding a browser in a Unity environment. Unfortunately the last time I checked none of them worked in Unity exported to UWP.
>
> As Patrick explained, it's not possible to spawn a XAML control into the Unity environment. It IS, however, possible to temporarily switch the user out of 3D mode and into 2D mode and display a XAML page which can host the WebBrowser XAML control.
>
> Switching between 3D and 2D has been confusing and challenging, but last week when I was in Redmond I worked with some folks on the HoloLens team and I have it working quite well. In fact I wrote a class you can all use in your projects to do this very very easily.
>
> The class is available right now as source. It's not entirely done yet because I still need to allow you to collapse (remove) the 2D view when you return to 3D mode and I still need to allow parameter passing to the 2D view when you switch to it. I will add those features soon and I will do a full blog post when I'm ready, but you can start playing with it right now.
>
> The class is called AppViewManager and you can download the source from my GitHub here.
>
> You need to capture the "default" view (which becomes the 3D view when Unity initializes) and you need to create a new 2D view. I do this on app startup in App.xaml.cs right before the code navigates to MainPage. The code is this:
>
> // Capture the "Main" view which becomes 3D when Unity starts var mainView = await AppViewManager.CreateFromCurrentDispatcherAsync("Main"); // Create a new 2D view for the "Purchase" page. var purchaseView = await AppViewManager.CreateNewAsync("Purchase", typeof(PurchasePage));
>
> In the example above, PurchasePage is a XAML page.
>
> Now you can either hold on to those two variables (mainView and purchaseView) or you can always get the views back later by name:
>
> var purchaseView = AppViewManager.Views["Purchase"];
>
> All you need to do to switch from 3D (Unity) to 2D (XAML) is call:
>
> await purchaseView.SwitchAsync();
>
> All you need to do to switch from 2D (XAML) back to 3D (Unity) is call:
>
> await mainView.SwitchAsync();
>
> Again, this code may evolve since I need to allow you to collapse the views and pass parameters, but it should be nearly identical when I'm done. I'm also planning to submit this as a pull request eventually to the HoloToolkit Unity but I need to write documentation and a sample before I can do that and I'm out of the office all next week for business travel.
>
> Hope this helps everyone!

Has anyone tried this but did not navigate first to MainPage.xaml? Will this work if you try to navigate first to a new xaml page?

I tried this one below but it does work too. It wont switch to Holographic space and even crashes.

http://www.jonathanantoine.com/2016/06/29/hololens-launch-a-3d-holograms-app-from-a-2d-xaml-app-and-going-back-to-it-uwp/

Has anyone experience the same?

CauchY · February 2017

@jeffgonzaga We're in the same boat. I'm trying to start with a XAML page and then switch to 3D/Unity when a Button is clicked. I've already tested several tutorials but it won't work.

This FitBoxIPD project by @jbienz looks promising but I haven't been able to get it going.

When I start with a XAML page, I cannot switch to the 3D-View:
rootFrame.Navigate(typeof(StartupPage));
If I try to load the MainPage for the 3D view, my 3D environment isn't loading (sleeping symbol). At the moment, I have no clue what's wrong since I CnP'ed everything and just removed the DevicePortal stuff.

Ok, there's one thing. In the MainPage.xaml.cs script, line 59:
appCallbacks.InitializeViewManager(); //doesn't compile, needs argument
Instead, I tried:
appCallbacks.InitializeViewManager(null);
appCallbacks.InitializeViewManager(Window.Current.CoreWindow);

jbienzms · February 2017

@CauchY and @jeffgonzaga

Even the FitBoxIPD app doesn't start in XAML. It actually starts in Unity and then switches to XAML if there are no cached credentials. It may not be ideal, but I recommend this approach. Unity does a LOT of stuff with the underlying plumbing that I don't fully understand and starting from XAML first doesn't seem to initialize that plumbing in the same way.

If someone figures it out, please do share here. But otherwise I recommend launching into Unity and just have your first scene empty with one behavior that transitions to XAML. You'll still see the Unity logo on load, but with an empty scene you should get to the XAML pretty quickly.

If you plan to run your app on tablet you might also want a little helper like the IsHolographic property here. Especially if you never plan to show Unity content in the tablet version. Then you can skip initializing Unity altogether.

Finally, if you do plan to run Unity in the Tablet version of the app, you might want to check out my Adept project. I haven't had time to work on it in months, but one of the cool features is the ability to run Unity as a XAML control, and the control helps with all kinds of interop issues like converting Tasks to Coroutines and back.

Best of luck guys. This is new territory!

jeffgonzaga · March 2017

Thanks @jbienzms, I finally got it working.

DiegoV · December 2017

@jeffgonzaga said:
Thanks @jbienzms, I finally got it working.

How?

How to develop 2D+3D hybrid apps (C#)?

Answers

Our Holographic world is here