Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Project - 480nm HoloViewer

Alex480Alex480 ✭✭
edited August 2016 in Projects

While waiting for the Holo Store update (it seems it is not that often) I decided to post some info here.

We have been working on a new 3D viewer for the HoloLens. Basically something like the 3D Viewer Beta but with more features and hopefully bit more friendly experience. The app is a general purpose viewer for 3D objects and scenes. While it is designed and optimized for scenes that are created in 480nm Studio, the viewer also provides additional support for two of the most popular exchange file formats: OBJ and FBX.

Some of the features:

Tools - Intuitive “pinch and move” set of tools to allow movement, rotation and scaling of models or complete scenes.

Voice commands - when using HoloLens, voice commands are more intuitive than ever and the viewer can take full advantage of the voice recognition system.

OneDrive - Support for loading models or scenes from OneDrive account. Intuitively and quickly place the files on desktop or other devices where you have instant access of those files on the HL. There is no need for wires or specialized upload procedures.

Complex shading - An advanced shader, capable of spherical harmonics, reflections and Fresnel effect that are applied at the pixel level which provides a realistic representations of true materials. Then the interface allows you to swiftly narrow down the exact combination of the properties at run time.

HDR Environments - 10 high dynamic range environment maps to provide the most realistic illumination of your model or scene. These images are courtesy of Christian Bloch (Blochi).

Spatial surfaces - One of the more powerful features of HoloLens is its ability to map the local environments, and we provided build-in visualization of these spatial surfaces as you navigate around your scene.

The app is available here:
https://www.microsoft.com/en-us/store/p/480nm-holoviewer-beta/9nblggh5127g

It is still in beta. Feedback and comments would be helpful.

Comments

  • DanglingNeuronDanglingNeuron ✭✭✭
    edited August 2016

    Cool! tried to use it but it crashes a lot. havent been able to load any model in it to view yet

    Healthcare IT professional by day - Indie GameDev for UWP and mobile platforms by night

  • Same here, models wont load from oneDrive.

  • Can you let us know which models (OBJ/FBX)? We tested it mostly on OBJs and few FBX files. If the models are in public domain it would help us to know where we could find them and repro the issues. The viewer will crash on very large models - about 100K polys is the safe zone for now for FBX, bit more for OBJ. That also seems to match HoloLens hardware. Thx for trying - it is already of great help!

  • Great viewer! After the 3D Viewer (original HoloLens launch app) failed to load my .fbx logo, your app worked well. A few things that I think could improve the experience:
    1. Pinch-dragging while using the rotate tool was hit & miss. While the model always rotated, it was difficult to get it rotating the way I intended. Tools like a bounding box with specific levers for rotation/scaling/moving like the Holograms app could help a lot.
    2. Related to #1, the axis origin for rotating the object (an extruded text logo) was not in the center of the model. I'll have to check again to see if the problem was with the fbx file by changing the object axis center in Cinema4D before exporting the fbx again
    3. After moving, rotating and changing colors, I suddenly lost the toolbox UI. I looked all over the room and finally had to bloom and relaunch. I have not reproduced this yet. Actually, at first I didn't even notice the toolbox as it was outside my FOV
    Great work and thanks again for sharing! Of note, I had to search in the store to find the app. When I clicked on the link, it sent me to a web page that indicated the app would not work for my device. I see this all the time for new apps that are not updated by Microsoft's store - I thought this was something that was known and being addressed.

  • Thx for the feedback! First, few comments regarding the crashes/hangs that have been spotted. I think I might have finally understood the issue and will address it properly soon. The FBX's SDK is not known for its speed and the average scene on a decent PC can easily take few seconds or more to load. On the HL device that is much longer and it is not unexpected to reach half a minute or so. That of course would appear as a hang. I have seen that in our testing but working with FBX for years by now I got used to it and didn't think much of (my fault). I will add some loading animation in to make sure something always shows in the viewer.

    The toolbox can be hidden and shown with the air-tap (pinch) gesture with the gaze outside the toolbox so it could "accidentally" get lost that way. Doing the air-tap anywhere will bring it back at that location.

    Some apps like to have the tools (toolbox) to "follow" the user and his/her gaze... I personally don't like it but with a bit of UI it could be done both ways so the user can choose the behavior.

    Rotation. Indeed, might need some improvements. A good way to provide more control is with additional UI/tools in the hologram/object space, so the "Holograms" app approach should work well.

    The Store. Yes, it seems it is somewhat stuck, but as outside devs not sure we can do much about it. Probably in a week or two I will be less happy about that, but for now I will take advantage of that to fix some of the issues before other people see them.

  • The latest update has just been published (1.1.5). I mostly focused on two key issues as mentioned so far. The hangs and the rotation complexity.

    Hangs. I moved all the FBX and OBJ loading code to a separate thread and kept the main loop running. In retrospect I should have done it that way a while ago... Anyway, the loading hologram will show in the meantime and it is animated to confirm all is well. It is fixed to the gaze so it will not be lost easily. I hope that solves the hangs reported so far.

    There is still a bit of delay during the transition, and it will be addressed at some other time. Should be minor.

    The rotations. The "ball" rotation model is quite universal but it is not precise - in fact it is hard to sometimes rotate to an exact orientation. As suggested, I have added the arrow tools around the object's bounding box. They appear/disappear based on the gaze so not all will show at once.

    Again, thx for the feedback.

  • @msrAR1510 said:
    Just tried the 480 nm Holoviewer app. Not bad, actually brought in an Obj file (but with no materials, only imports the obj), and previously, I had imported an FBX file. Movement, rotation, and sizing work, but are slow and with a delay of a few seconds after moving your hand/fingers. I like the materials, lighting, and color options. Enabling spatial mapping shows a mesh as if it was trying to map, but then the hologram disappears, and the mapping never seems to be complete. The holograms (in my case, a 3d model) do not seem to be locked into position very well, so when you move, they seem to move some also, so tough to walk around them and view from different angles. Is there any documentation for this app?

    Thx for the feedback. It is very valuable at this time and really helps me to make it better.

    • When it comes to the OBJ files the materials are always an issue as they are separate files and the only way we can easily load files on HoloLens at this point is over OneDrive, which I believe can only handle a single file. Handling ZIP archives would allow the user to prepare all the resources into one file so that would be a solution to this problem in the future.

    • These days my focus is on FBX materials and textures which FBX allows to embed. That makes it bit easier as all the information is stored in a single file. I have it almost ready and will try to publish next week.

    • The spatial mapping is occluding to the model (but not to the UI console). I will have to test partial occlusion (alpha) to see how that appears - I believe just a partial visibility would be of great help to hint the user where the model is and that it might need to be moved.

    • The slowdown is an issue I need to narrow, should be temporary. I have seen it as well for larger scenes.

    • The locking issues is something I was unaware - I will certainly need to test it in larger rooms - my dev room has only few square feet of empty space left, so not much walking around...

    Once again, thank you for your comments.

  • Thanks for the quick responses, this is great! Regarding a comment above about files not loading from OneDrive, note that there are two parts to the load...one is the download, but then you have to tap the "open" button in the lower right (the typical cloud with arrow) to open the model. I did not notice this my first few attempts, and thought files were not downloading.

  • The latest update is now available in the store - version 1.1.6.0:

    • FBX Importer - I added support for importing materials and textures. I know it conflicts now a bit with the previous material settings and colors as controlled by the UI - so if the model has a material that material will be used and the UI settings will be ignored. If the model has no material then as before - the user can specify it at run time. Not ideal, but should work for now.

    • FBX Importer - I added support for importing vertex colors. It has been a specific feature request, but I believe it will be of benefit to everyone - for example baked AO. The sample protein model below has been provided to me courtesy of James Tyrwhitt-Drake from NIH/NIAID.

    • Pivot point. It is now based on the center of the model's bounding box rather than the origin point. Ideally they should be the same but in many cases the models were not centered and the tools would not work too well in such cases. Now all models should be exactly centered and the tools work as expected.

    As always, let me know if you have a feature request or any feedback - even a bad one - over the years I have learned how to handle it all.

  • Hi Alex480, could I have a short talk with you via Skype for some special questions concerning your app? Thanks! Use HoloLorenz to locate me on Skype.

  • Version 1.2

    I'm really excited to release the next update - version 1.2. It should be already available in the store, or as update if you have previously downloaded the app.


    But first the story of hangs and failures to start.
    With the release of the previous versions I was finally able to start receiving the crash and hang reports. What has immediately got my attention was the unproportionally high percentage of hangs relative to crashes - almost 10 to 1. As anyone who has been developing for a while knows that is really rare and unexpected - especially since the out of memory conditions that are likely on HoloLens would certainly lead to crashes and I could not think of anything that could hang... Well, ironically it took a somewhat negative review feedback to hint me something was not right - that the app would simply fail to start at all. As developers of course we never experience these things (just our luck). Anyway, it is true that the first time the app starts it can take about 20 seconds to build all the caches and so on and in that time it was not that UI-responsive. What I didn't know is that while on desktop apps that is not great but overall fine, on HoloLens Windows is much less forgiving and simply kills the app in question (no message at all). So in this version I took extra care to show the UI as quickly as possible and move as many longer calculations to background async threads. I'm highly optimistic that that should solve the unexpectedly high number of timeout hangs and if you have tried the viewer in the past and it failed to start I encourage you to give it a try again (other bugs and crashes are still expected)


    What's new:
    * Completely new, cleaner and faster user interface. While I liked the previous somewhat 3D-based UI it was slow and very difficult to enhance or adjust - with every change I had to regenerate the geometry. So from this version on, the UI is based on a proper custom windows system with all the regular controls. It is much faster and easy to adjust so adding new features should be trivial. The windows can be moved by tapping the title bar, and adjusted with controls at the frame perimeter.

    • Support for skeletal animations with FBX files. That was somewhat the last major missing piece to the FBX importer. The key to get that working was of course performance. Now with many optimizations and vertex shader based skeletal animations the animated models should render as fast as the static ones. Finally the 3D Viewer's Shark looks and animates the same (somewhat my personal goal to match the other viewer in terms of features).

    • Support for 360 degree (360°) environment panoramas. That was an easy addition, so why not. To simplify the UI the only supported images are in the equirectangular panorama format (the 2:1 rectangular images). I believe these are the easiest and most common, but if you would like to see support for other forms (the cubic, angular map, balls, and etc.) let me know. The code is all there, I would just need to think of some easy UI solution.

    • Physically-based sky light environment. Still to be improved - since as it is now, I'm not that happy with it yet.

    • Some minor features like reset to restore the transformations of the models and flags to control textures, environments, spatial surfaces and vertex colors. Also the play/pause button has been added.

    Quick note: While most of the shaders are already prebuild if the app appears "hang" after loading a new scene give it 10 or so seconds - it might happen that a rare type of material forced a new shader to be compiled.

    If you like and enjoy this viewer, please rate us and leave a feedback at the Microsoft Store.

    We also would love to hear from you with regard to feedback, bugs or ideas. Please contact us at:

  • Version 1.2.4

    Just posted the new version to the store. Mostly bug fixes, but also some new features. I finally finished the physical sky the way I wanted it. While neat on its own it was mostly a test ground for my additional UI controls.

    What's new:

    • Fixed crash bug on startup. Going async can sometimes lead to unexpected issues and that was one of them with the audio synthesizer resulting in about 10% probability of crash on startup.
    • Completed the Physical Sky window. A complete location and time system allows the user to pick expected illumination for any location and time. Also a test ground for somewhat more complex UI elements.
    • Added ground plane and shadows. The ground plane was needed for the shadows to make sense. Time will show how useful these features are. The shadows are unfortunately very expensive time-wise resulting in noticeably slower frame-rates, so they are disabled by default and have to be enabled in the UI.
    • Optimized the shader caching system - this mostly affects me as I no longer have to manually prebuild all the possible shaders - makes it much easier to adjust the existing shaders and deploy the app.

    The completed physical sky dialog:

    The new ground plane and shadows:

    Feedback. Don't hesitate to let me know of any issues or ideas. The HoloLens community is still very small and as we don't get many comments, the few we get are taken seriously and much appreciated.

    Future work. For the next release I'm mostly considering adding support for additional file formats and starting working on a tree/list view to allow for individual object selection and adjustments as right now the whole scene is affected in material or transformation changes.

  • Version 1.2.5

    We are excited to publish another update. A lot of work went into it - mostly spend on supporting new file formats and optimizing the rendering engine.

    New file formats: JT and STL

    • JT (Jupiter Tessellation) is an ISO-standardized 3D data format that is very popular in the CAD industry. At this point the supported versions are 8 and 9. As a developer I have to say it was one of the most complex formats I worked on - it uses multiple forms of compression and various encodings, and to complicate it more there are substantial differences between version 8 and 9. Anyway, I would like to be sure it works well - if you encounter scenes that crash or fail to load please e-mail them to us.


    • STL. Another popular format for CAD models. Very simple and certainly with limited features, but sometimes that is exactly what is needed. Both ASCII and Binary files are supported.

    Optimizations

    We have spent many hours optimizing the engine handling of scenes with multiple objects as they tended to be quite slow. They should render noticeably faster now. At the same time the shaders are still the same, so features like shadows still tend to be major slowdown.

Sign In or Register to comment.