Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

How to achieve constant 60 fps?

Hello everyone.
I work in a company which is developing for HoloLens since 2nd wave (it is not our main project - working like 3h per week) and one of my main dificulties since then is the fact that I cannot guarantee 60 fps on any scene that I created so far (keep in mind that I'm not - and none of our developers are - game developers).
Our project consist in taking some .OBJ / .FBX / .SKP 3D-modeled files, get a 3D designer to do some mesh reducing work and bring it into HoloLens into a scene.
On Unity we change all shaders to use the HoloToolkit shaders, the toolkit's camera, toolkit's FPSCounter and Unity is set to fastest quality settings.
After researching (a lot) about Unity shaders and Hololens performance issues, I ended up trying to address those performance problems with Unity's frame debugger and Visual Studio's GPU debugging.

With Unity frame debugger:

  • It seems to be nothing wrong with rendering order for objects on the scene. Is there any more useful data that I can parse on this tool?

With Visual Studio GPU debugging:

  • This amount of debugging data is overwhelming! I just tried to understand what all that data means...
  • Got 1 60 fps capable frame and pressed the "capture frame" button.
  • Then I found a object that makes my frame time drop and captured that frame.
  • In a comparison between the frames I ended up with a higher amount of "Draw Calls" on the problematic frame. But I just don't know what I can do to make it better.
  • The "Camera.ImageEffects" call is always the most long call in the frame (about 1~1.8ms), twice per frame - on both good and problematic frames.
  • Is there any more data that I'm missing at this tool?

Can any of you share experiences or tips for the resolution of this issue?

Some data for you guys:

  • One of the problematic objects have 498 triangles, 810 verts. I think the problem has nothing to do with poly count... Some of other objects are more complex and they're ok.
  • When disabling all of my scripts the behaviour won't change.
  • I'm using Unity 5.6.0b1. I used to try Unity's special edition for HoloLens with the same issues.
  • HoloLens emulator is not able to reproduce the issue. It happens only on the device.
  • GPU debugging makes the app run at 30fps but the frames still drops.

References:
https://forums.hololens.com/discussion/3296/maintain-60fps
https://forums.hololens.com/discussion/1652/holograms-moving-when-it-doesnt-supposed-to

Comments

  • It's hard but there are number of things you can do to help promote 60 fps. This is general advice if using unity or developing with your own engine:

    Use instancing if you are rendering multiples of the same mesh.
    Use vertex lighting and use lower precision types such as short, half and half view when calculating light and speculars.
    Keep your pixel shaderr as simple as possible.
    In unity, stay away from standard shader. Use unit or vertex lit.
    Prebake textures for detail , shadows, lights.
    Turn off physics and shadows in unity
    Set unity graphics quality at fastest
    Create mip maps and keep texture size under 2048

    Turn off aa and use bilinear filtering
    Go easy on the gc. Minimize the collection.

    These are basics to get you started and unfortunately a lot of these are topics on their own.

    http://www.redsprocketstudio.com/
    Developer | Check out my new project blog

  • @AmerAmer said:
    It's hard but there are number of things you can do to help promote 60 fps. This is general advice if using unity or developing with your own engine:

    Use instancing if you are rendering multiples of the same mesh.
    Use vertex lighting and use lower precision types such as short, half and half view when calculating light and speculars.
    Keep your pixel shaderr as simple as possible.
    In unity, stay away from standard shader. Use unit or vertex lit.
    Prebake textures for detail , shadows, lights.
    Turn off physics and shadows in unity
    Set unity graphics quality at fastest
    Create mip maps and keep texture size under 2048

    Turn off aa and use bilinear filtering
    Go easy on the gc. Minimize the collection.

    These are basics to get you started and unfortunately a lot of these are topics on their own.

    I understand what you are saying, but all of these tips have been cited on other posts and we tried almost everything without success, unfortunetly.

  • edited January 2017

    I would agree with @AmerAmer,
    To add on I would take advantage of multi threaded shaders, and use background threads. An example can be seen here.

    And here

    Now I know these examples are DirectX C++, however you may be able to extract the ideas and concepts out of these projects. The ImageEffects is a special DirectX Shader which appears to be causing your issue. Lookup a way to disable this shader or reimplement it, or rethink that lagging section to see if it's even needed. 

    Dwight Goins
    CAO & Founder| Independent Architect | Trainer and Consultant | Sr. Enterprise Architect
    MVP | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer
    http://dgoins.wordpress.com

  • Want better and more consistent FPS?

    Go a different path than heavyweight Unity. Once you drop the dead weight of Unity everything runs faster and more consistently.

  • You'll have to debug your scene even further focusing on your shaders. Unity tries to stabilize the frame rate by dropping down to 30 even if you may be running 45+ if there is to much variation. Look for draw calls and try to lower that number as much as possible. Verify that unity is set to use stereo instances.

    http://www.redsprocketstudio.com/
    Developer | Check out my new project blog

  • One thing I'd like to add to what's already been shared above is just really put some care into your artwork. In my experience so far at Microsoft working with customers, this is where I see the biggest performance challenges arise. Many times I see people attempting to take high poly objects from other projects (especially in architecture or mechanical engineering) and "downsize" them for HoloLens. And many times this just doesn't quite go as planned.

    When it comes to artwork and performance, the HoloLens (or any mobile GPU for that matter) really needs to be a consideration from the beginning. For really high performance we need to look at techniques commonly used in the mobile gaming industry. Things like modeling both high poly and low poly versions, then baking the illusion of detail from the high poly into normal maps that we display on the low poly.

    There are some tools that can help with this. Simplygon has been one of the best I've seen at this so far. Simplygon also helps with another common problem I've seen too: the total number of sub objects. If you expand your imported object in Unity and see 10s or even 100s of sub objects you are likely going to run into performance problems. Without getting too deep into the technical issues, you're likely stressing the Unity scene graph and rendering loop. You may also be inadvertently impacting the number of draw calls. Simplgon has a feature to create "proxy" objects, where it basically attempts to combine multiple individual pieces of geometry into a single object and then attempts to unify the materials. This can greatly speed up performance.

    These are things that can help you, but unfortunately there is no magic bullet and there is no substitute for having an awesome 3D artist on staff.

    I have been working closely with a good friend and 3D artist on some guidance in this area. We will be releasing a high performance and highly detail HoloLens demo soon as well as an accompanying article. It will be on roadtoholo.com (linked in my signature). I'll try to remember to come back here and post a link when that article goes live.

    Best of luck. I really hope you are able to resolve your performance bottlenecks.

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

  • AmerAmerAmerAmer ✭✭✭
    edited January 2017

    I just thought of something. The way we debugged our framerate fluctuations was watching the framerate as we moved around with the device. Try to see if an individual meshe or ui component is causing the framerate to drop. Start with a point where you are either looking at nothing or something that gives you a steady 60 fps. Typically an empty spot away from all holograms is a good start. Slowly move your head to fill the viewports with one component at a time until you find the one that drops your framerate. Once you identify the component, start looking only at certain parts of it and see where the framerate drops. Sometimes its just a spot where there is a ton of vertexes overlapping. This is usually a pixel overdraw problem. Meaning that your pixel shader is being called over and over for the same set of pixels because of overlapping polygons. So removing internal polygons to eliminate the overlap. You can continue testing this by walking as far away from the mesh as possible and see if the framerate increases. If the fps tanks up close but is ok far away then thats a pixel shader issue. If its at any point, then your mesh has some issues or your vertex shader is doing way too much work. Make sure you don't have overlapping vertexes, or splices in your mesh causing weird rendering issues. You can always take your mesh to maya or blender and remove duplicate vertices

    http://www.redsprocketstudio.com/
    Developer | Check out my new project blog

  • @jbienzms said:
    One thing I'd like to add to what's already been shared above is just really put some care into your artwork. In my experience so far at Microsoft working with customers, this is where I see the biggest performance challenges arise. Many times I see people attempting to take high poly objects from other projects (especially in architecture or mechanical engineering) and "downsize" them for HoloLens. And many times this just doesn't quite go as planned.

    When it comes to artwork and performance, the HoloLens (or any mobile GPU for that matter) really needs to be a consideration from the beginning. For really high performance we need to look at techniques commonly used in the mobile gaming industry. Things like modeling both high poly and low poly versions, then baking the illusion of detail from the high poly into normal maps that we display on the low poly.

    There are some tools that can help with this. Simplygon has been one of the best I've seen at this so far. Simplygon also helps with another common problem I've seen too: the total number of sub objects. If you expand your imported object in Unity and see 10s or even 100s of sub objects you are likely going to run into performance problems. Without getting too deep into the technical issues, you're likely stressing the Unity scene graph and rendering loop. You may also be inadvertently impacting the number of draw calls. Simplgon has a feature to create "proxy" objects, where it basically attempts to combine multiple individual pieces of geometry into a single object and then attempts to unify the materials. This can greatly speed up performance.

    These are things that can help you, but unfortunately there is no magic bullet and there is no substitute for having an awesome 3D artist on staff.

    I have been working closely with a good friend and 3D artist on some guidance in this area. We will be releasing a high performance and highly detail HoloLens demo soon as well as an accompanying article. It will be on roadtoholo.com (linked in my signature). I'll try to remember to come back here and post a link when that article goes live.

    Best of luck. I really hope you are able to resolve your performance bottlenecks.

    This is the best post I've seen so far. Very enlightening.

    I'll work with our 3D artist based on your response.
    Thank you very much.

  • @AmerAmer said:
    I just thought of something. The way we debugged our framerate fluctuations was watching the framerate as we moved around with the device. Try to see if an individual meshe or ui component is causing the framerate to drop. Start with a point where you are either looking at nothing or something that gives you a steady 60 fps. Typically an empty spot away from all holograms is a good start. Slowly move your head to fill the viewports with one component at a time until you find the one that drops your framerate. Once you identify the component, start looking only at certain parts of it and see where the framerate drops. Sometimes its just a spot where there is a ton of vertexes overlapping. This is usually a pixel overdraw problem. Meaning that your pixel shader is being called over and over for the same set of pixels because of overlapping polygons. So removing internal polygons to eliminate the overlap. You can continue testing this by walking as far away from the mesh as possible and see if the framerate increases. If the fps tanks up close but is ok far away then thats a pixel shader issue. If its at any point, then your mesh has some issues or your vertex shader is doing way too much work. Make sure you don't have overlapping vertexes, or splices in your mesh causing weird rendering issues. You can always take your mesh to maya or blender and remove duplicate vertices

    This is a very good advice too, thanks!

  • jbienzmsjbienzms mod
    edited January 2017

    @idealsoft said:
    This is the best post I've seen so far. Very enlightening.

    I'll work with our 3D artist based on your response.
    Thank you very much.

    Thank you very much for the kind words.

    One other thing I want to mention is that it's critically important that you're continuously updating the stabilization plane. The stabilization plane tells the device what part of the scene is the most important for rock solid world locking and tracking. If this is not set or is set incorrectly, you may see what appears to be drifting or jumping, which are behaviors commonly associated with bad framerate.

    If this sounds confusing, don't worry. It's pretty easy to do correctly. Here are the steps:

    1. Make sure that you've included one (and only one) GazeManager script somewhere in your scene.
    2. Include one (and only one) StabilizationPlaneModifier script somewhere in your scene.
    3. Make sure StabilizationPlaneModifier.SetStabilizationPlane and StabilizationPlaneModifier.UseGazeManager are both set to true (true is the default for both settings but it's good to verify).
    4. Last and most important: Any hologram that the user will spend a notable amount of time focusing on should have a collider that surrounds its basic shape. This would ideally be a BoxCollider, SphereCollider or CapsuleCollider for performance reasons, but if you have a strong need for a tight fit of the object you can use a MeshCollider at the cost of some performance.

    Let us know what you find!

    Our Holographic world is here

    RoadToHolo.com      WikiHolo.net      @jbienz
    I work in Developer Experiences at Microsoft. My posts are based on my own experience and don't represent Microsoft or HoloLens.

Sign In or Register to comment.