Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

Hololens in Camera (Resolution Differences when Taking a Photo and Using Unity's Camera GameObject)

I'm trying to use the Hololens camera to take a photo, run an OCR on the photo, and then place tags in world space based on the pixel coordinates I get from the OCR. However, I find that the world coordinates are little bit off because of the different image resolutions I'm using.

When I take a photo using the Hololens camera, I take the photo in 2048x1152. I send this to the OCR and get back pixel coordinates. During this time, I position a second camera (not the main camera) to where I took the picture. Then, I raycast from this camera using Unity's Camera.ScreenPointToRay function. However, the Unity camera is 1280x720. Even if I scale the pixel coordinates from the 2048x1152 resolution to the 1280x720 resolution, the tags are still off in world space. It seems like the Hololens is cropping the photo instead of scaling since nothing can be raycasted outside the 1280x720 resolution. I think this is the issue because when I take the photo in 1280x720, everything shows up correctly. However, I'd like to make the field of view bigger so using 2048x1152 would be ideal.

Unfortunately, Unity cameras have read-only pixel width and height so I can't change the resolution. I tried to do Screen.SetResolution(2048, 1152, false) but that didn't work so well. I look at https://developer.microsoft.com/en-us/windows/holographic/locatable_camera, but didn't find much on changing the resolution of the Unity camera.

Has anyone encountered this problem or have suggestions? Thanks!

Answers

  • AmerAmerAmerAmer ✭✭✭
    edited February 2017

    Its the lense. I haven't gotten higher than 720p on video and I believe image as well. You might have to scale the resulting image before you process it. I remember talking to Unity editor/engine developers about the LocatableCamera and they had a bunch of bugs in the pipeline to fix but that was during their 5.5 early beta. They might have fixed it by now.

    1280x720 45deg (default mode)
    2048x1152 67deg Highest resolution still image
    896x504 896x504 896x504 48deg Low power / Low resolution mode for image processing tasks

    http://www.redsprocketstudio.com/
    Developer | Check out my new project blog

  • Hi jonathanhuang,
    i'm developing an app similar the your. I had the same your problem and understand that it can't be solved because it's not possible to raycast outside the 1280x720.

    You said that using the 1280x720 resolution all works fine, but not for me. Let explain.

    In my case I'm talking about the object detection and not OCR, so the hololens wait for 4 coordinates for each bounding box.

    Step:
    1) Take the picture
    2) Send to a server for elaboration
    3) Receive the 4 coordinate
    4) Evaluate the center of the bounding boxes
    5) Raycast the center

    What happen is that for the object that are in the center of the image it works fine. So the ray is casted correctly in the center of the bounding box, but for the object that are on the right or on the left, the raycasting is off. In particular the ray are casted slightly towards the center.

    For understanding:

    • if an object is on the left, the ray is casted slightly further to the right
    • if an object is on the right, the ray is casted slightly further to the left

    The same for top and bottom.

    Can you help me to explain this behavior?

    Thanks a lot.
    Ciro Varriale

  • XyyXyy
    edited April 2018

    Hello.Can I ask you the solution to do object detection? Do you use OpenCV ? or something else? Can you give me some detail? Thank you!

  • Hey,[email protected]

    what does these "4 coordinates" you mentioned above refer to ?

Sign In or Register to comment.