XAML-Visual Tree and HoloLens® 3D-Volumetric View

AdrianWenz

Currently, the 2D-Xaml view seems to be deactivated when a 3D-volumetric view is active at that time inside the same application. While the UI thread is still running, some functionality, like for example 'RenderTargetBitmap.RenderAsync(...)' is blocking:

await CoreApplication.MainView.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () =>
{
      RenderTargetBitmap Bitmap = new RenderTargetBitmap();
      await Bitmap.RenderAsync(PanoramicImage);  <= Blocking here
      ...
});

This is a very limiting situation because data only obtainable from the 2D-Xaml view (e.g. as 'IBuffer Bitmap.GetPixelAsync()') cannot be passed to and rendered in the 3D volumetric view.
Can we expect an additional API for that situation?
Thanks

Dwight_Goins_EE_MVP

Can you explain a little more about what you're trying to do?

Rendering in 3D is not as simple as passing a buffer of 2D pixels and displaying them. You have to convert the 2D pixels to 3D, and be aware of lighting, camera, view, rotation and etc, all in 3D World space coordinates.

I think your approach is wrong. It's probably better to think in terms of 3D first, where you position objects/models and meshes of what you want drawn in world space, and then create 2D Textures (bitmap pixels), and draw then on top of the 3D Models/Meshes.

In otherwords, treat the world or application like a 3D app, with 2D textures drawn out.

AdrianWenz

@Dwight_Goins_EE_MVP Well, the scenario is similar to the one outlined in the thread https://forums.hololens.com/discussion/comment/9562#Comment_9562, however the proposed solution does also block and hence doesn't do the job.
@HoloLens_Forum_Admin I wonder if there is a (future) API that keeps the XAML-visual tree fully functional while running in a 3D-volumetric view. That would unfold enormous possibilities for the HoloLens®.

Simon

@AdrianWenz said:
@Dwight_Goins_EE_MVP Well, the scenario is similar to the one outlined in the thread https://forums.hololens.com/discussion/comment/9562#Comment_9562, however the proposed solution does also block and hence doesn't do the job.

I initially shared your frustration with the fact that RenderAsync() blocks without end, but I was only interested in using XAML for the WebView control, which I found has a CapturePreviewToStreamAsync() method that does return the image in a timely fashion, though I'm guessing that wouldn't help you if you wanted to use other XAML controls.

Have you tried running a XAML program on a separate computer and streaming the pixels over the network? That's the only other option I can think of.