Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

Holograms moving when it doesn't supposed to

Hello,
Our company is new to Unity development and we are making some tests with HoloLens in the last weeks.
One behavior we noticed is that in one simple scene with no Spatial Mapping, no complex scripting (no transition on Update() methods), it keeps lagging/moving when moving the camera towards its position. None of the objects has a collider nor a rigidbody attached.
It won't happen in the emulator neither on other apps - like Holograms.
I've made a video to show the problem.
YouTube video

Did anyone had this problem or know how to solve it?

Thanks in advance

Clayton Poitevin
IdealSoft - Brazil

Best Answer

  • AlexDAlexD ✭✭✭
    edited July 2016 Accepted Answer

    If you are at 30FPS you found your problem.

    As long as you are not at a stable 60fps your holograms will drift and not be stable enough. Essentially at 30fps the device does not have enough time to respond to your head movements and correct the positioning of the holograms to account for that and create a perfect illusion.

    There are a couple of really good articles about performance on the HoloLens documentation site here and here

    Remember, the asset store is not optimized for HoloLens consumption so more often than not you cannot use the assets straight up in your app. But, you can simplify your models that you bought using 3rd party software. We've used MeshLab but you can search for others.

    Lastly, the easiest way to inspect your poly count in your scene is to use the 'Stats' window from Unity. Run your scene in the Unity Player then in the Game view enable the 'Stats' window. it will show you how many vertices and triangles are in view (not in the whole scene, but in the view which is what you need).

    Hope this helps.

Answers

  • Hi Clayton,

    Are you expecting the model you are rendering to stay in the same place rather than move along with you like it seems to be?

    Would you be willing to share your project with me so I can take a look? My (often wrong) gut has a couple guesses:
    1. The model has a lot of triangles and/or is being rendered with some complex shaders.
    2. the model is VERY VERY VERY large and is being rendered several Kilometers away from you.
    3. The Tagalong script is on the model.

    ===
    This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.

    (Daddy, what does 'now formatting drive C:' mean?)

  • Thanks for the quick response, Patrick.
    This Unity project is now too large for me to share I think - it has all the test scenes we did since the beginning of our tests (6~7GB). Is there any way to share the project on a smaller size that I'm not aware of?
    I'll respond to your guesses and add more information to see if you can help:
    1- Some models on the scene do have a lot of triangles. I used both HoloToolKit Standard-Fast and Unity's Standard shaders and both seem to have the same output.
    2- I think it is not the case - the model is designed to be a 1:1 proportional in size to one of our rooms here at our company. Usually we can walk around the furniture presented on the video, but for demonstration purposes I made the camera initialize far from the model. On both distances the behavior is the same.
    3- When disabling all scripts in the scene the jittering/lagging do occur.

    The project settings are set to the fastest graphics.
    I noticed that this happens a lot faster when a wall is too near the Hololens, too.

  • What happens if you take just that model and put it in a new unity project?

    ===
    This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.

    (Daddy, what does 'now formatting drive C:' mean?)

  • BTW, some instability can be caused by not setting the focal plane. This is especially noticeable if you are closer than 2m to the Hologram. You might try adding a script like this to the model:

    void Update()
        {
            if (IsTargetVisible())
            {
                HolographicSettings.SetFocusPointForFrame(gameObject.transform.position, -Camera.main.transform.forward);
            }
        }
    
        private bool IsTargetVisible()
        {
            // This will return true if the target's mesh is within the Main Camera's view frustums.
            Vector3 targetViewportPosition = Camera.main.WorldToViewportPoint(gameObject.transform.position);
            return (targetViewportPosition.x > 0.0 && targetViewportPosition.x < 1 &&
                targetViewportPosition.y > 0.0 && targetViewportPosition.y < 1 &&
                targetViewportPosition.z > 0);
        }
    

    ===
    This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.

    (Daddy, what does 'now formatting drive C:' mean?)

  • another thing (sorry for those of you who go crazy when someone double posts):
    does your model have any Skinned Mesh Renderers or animations? You might want to try the GPU profiler (start debugging with Alt+F5).

    On the device portal page on the performance tab the FPS would be interesting to know as well.

    ===
    This post provided as-is with no warranties and confers no rights. Using information provided is done at own risk.

    (Daddy, what does 'now formatting drive C:' mean?)

  • Patrick,

    Here's the feedback of the things I tried based on your posts:
    1- Put the focal plane script on the base GameObject. It seems it hasn't changed anything. What supposed to happen?
    2- Did the GPU profiler on the project. The scene shows up completely flickering.
    Video
    3- Created a new project and put the models on it. Same behavior seems to occur.
    My models do have complex mesh renderers (purchased from the Unity Assets Store), but on the "stats" box on unity the Visible Skinned Meshes is set to 0. Here is the link to the new project I created (without the holotoolkit shaders):
    Link

    I think maybe HoloLens won't be able to render the scene due the poly count. :neutral:
    I don't know if I'm doing anything wrong but it is such a simple project...

  • Alec_HAlec_H ✭✭

    @idealsoft I've noticed the same problems with some data I've put in for modeling. The solution I've found for this is click on the asset you are trying to use in the asset pain. In the inspector window click on Model tab, then there is a mesh compression setting. Try cranking that up to high. This compresses the mesh a bit and gets rid of some of the detail, but makes it easier on the HoloLens. I am assuming the issue is due to no Gpu or not a super powerful Gpu on the HoloLens. Hopefully they will get better with time, but for now this is a fix I found.

  • @Alec_H thanks for the help.
    Your tip helped us to keep a higher FPS on our scene, but the positioning still seems wrong. I did another video to show how a hologram that seems to be in a place moves far from the camera when it gets closer to it.
    Video

    Any clue? Do you guys think I need to try save the room and put the objects right with spatial mapping to achieve what I want?

  • Alec_HAlec_H ✭✭
    edited July 2016

    @idealsoft I maybe wrong, but it looks to me the scan of that room just isn't very accurate. To my understanding you can force the Hololens to map a room more accurately by walking around and air tapping. This makes sure all of the borders of the room are better mapped. I think the issue for you is when you get super close the mapping in that corner wasn't the best and the Hololens is getting confused on where the boarders actually are. So I think you should try manually scanning the room for best and most accurate results. I've noticed too if you just initially walk into a room sometimes the Hololens doesn't always get the corners super well. This could be due to lighting differences and so many other variables. These variables also change the closer you get to a corner. Hopefully this helps!

  • @Alec_H I understood what you said. I just went to out test room, did the airtapping on every wall, ceiling and floor, and the same thing happens (don't know if this menu spatial mapping affects the application, but still).
    The room also have some lighting (3 windows) but if I put other app on the same room it works flawlessly.
    If someone have the time, I put my project link on OneDrive some comments above to see if anyone with more experience than me can reproduce and help us solve it.
    Thanks for the comments!

  • AlexDAlexD ✭✭✭

    Did you check the frame-rate of your application? Holotoolkit has a prefab called FPSDisplay that you can just drop in your scene and see check your performance.
    The only times when I've experienced serious hologram drift was when I was when my framerate was around 30fps. Ideally your app should run at 60fps and in that state I've had perfect hologram positioning.

    One thing I noticed about models though is that it's not necessarily only the poly count that you need to look for but also the Polygon density. I once had a very dense object at 40k and it would still make my framerate drop to 30. Might be worth to know for when you're optimizing your models.

    If you upload your code to OneDrive I might have some time today to take a look.

  • @AlexD My example code is few comments above!
    Project

    When looking to some models on my scene the FPS count drops to exact 30 frames.
    All models in the scene were bought from Unity assets store. How can we check for polygon density in this case?
    We're new to Unity and things like model optimization is something that we still have a lot to learn.

  • AlexDAlexD ✭✭✭
    edited July 2016 Accepted Answer

    If you are at 30FPS you found your problem.

    As long as you are not at a stable 60fps your holograms will drift and not be stable enough. Essentially at 30fps the device does not have enough time to respond to your head movements and correct the positioning of the holograms to account for that and create a perfect illusion.

    There are a couple of really good articles about performance on the HoloLens documentation site here and here

    Remember, the asset store is not optimized for HoloLens consumption so more often than not you cannot use the assets straight up in your app. But, you can simplify your models that you bought using 3rd party software. We've used MeshLab but you can search for others.

    Lastly, the easiest way to inspect your poly count in your scene is to use the 'Stats' window from Unity. Run your scene in the Unity Player then in the Game view enable the 'Stats' window. it will show you how many vertices and triangles are in view (not in the whole scene, but in the view which is what you need).

    Hope this helps.

  • Alec_HAlec_H ✭✭

    @idealsoft I had now issues using the files you supplied with it moving on me. I did delete the FPS object and it works just fine on my Hololens. Good luck and hope you guys figure it out.

  • @Alec_H said:
    @idealsoft I had now issues using the files you supplied with it moving on me. I did delete the FPS object and it works just fine on my Hololens. Good luck and hope you guys figure it out.

    Did it ran smoothly at 60 fps the entire time?

  • Alec_HAlec_H ✭✭

    @idealsoft I'm not sure the exact frames because it was on my Hololens, so I don't think it showed the exact FPS, but yes it looked smooth and just fine to me.

  • AlexDAlexD ✭✭✭

    You can check the FPS on HoloLens using the device portal.
    It's important to check the FPS when the offending model is in the view.

  • An update:
    I did a lot of modifications to our project for testing purposes.
    Every complex model was replaced by a simple cube that has the same size, without any material, colliders or physics.
    No spatial mapping enabled. No FPS counter on the screen.
    Unity shows less than 600 triangles in the scene.
    Well, after all that work, I still couldn't achieve 60fps when looking to some of the cubes I placed in the scene (I did what @AlexD suggested and checked FPS on the device portal - it clearly drops from time to time, to 30~50fps with 0~14 VBlank/sec on our project and other apps like GalaxyExplorer).

    Running some other store apps does have the same behavior of running at 30~50fps.

    For testing purposes too, I've created a DirectX HoloLens project on VS2015 and started the spinning colored cube demo. This project did 60fps at the entire time, no matter what position I were or where I walked.

    Am I missing something on Unity's configuration or on the project handling?
    I'm running Unity 5.4.0b24-HTP with VS2015 Update 3.
    Here I'll link my new test project to see if you guys can help me out.
    https://1drv.ms/u/s!AgCbcxAd5uWfjiIecFN6hV1CJ29b

    Thanks in advance.

  • @idealsoft Hi there, did you ever identify the problems with your project that impacted performance?

  • @idealsoft,
    If you're not achieving 60fps with simple cubes, then check the quality settings for your project. In Unity, select 'Edit > Project Settings > Quality' and then change the default settings from 'Fantastic' to 'Fastest'. This will disable shadows, which can help boost performance on the HoloLens.

    Also, if the materials on your models are currently using Unity's 'Standard' shader, you should change them to use a more mobile-friendly shader instead. Creating custom shaders, using shaders provided by the HoloToolkit, or changing to Unity's Mobile or Legacy shaders (VertexLit is pretty efficient) will also improve performance.

Sign In or Register to comment.