Hololens in Camera (Resolution Differences when Taking a Photo and Using Unity's Camera GameObject)
I'm trying to use the Hololens camera to take a photo, run an OCR on the photo, and then place tags in world space based on the pixel coordinates I get from the OCR. However, I find that the world coordinates are little bit off because of the different image resolutions I'm using.
When I take a photo using the Hololens camera, I take the photo in 2048x1152. I send this to the OCR and get back pixel coordinates. During this time, I position a second camera (not the main camera) to where I took the picture. Then, I raycast from this camera using Unity's Camera.ScreenPointToRay function. However, the Unity camera is 1280x720. Even if I scale the pixel coordinates from the 2048x1152 resolution to the 1280x720 resolution, the tags are still off in world space. It seems like the Hololens is cropping the photo instead of scaling since nothing can be raycasted outside the 1280x720 resolution. I think this is the issue because when I take the photo in 1280x720, everything shows up correctly. However, I'd like to make the field of view bigger so using 2048x1152 would be ideal.
Unfortunately, Unity cameras have read-only pixel width and height so I can't change the resolution. I tried to do Screen.SetResolution(2048, 1152, false) but that didn't work so well. I look at https://developer.microsoft.com/en-us/windows/holographic/locatable_camera, but didn't find much on changing the resolution of the Unity camera.
Has anyone encountered this problem or have suggestions? Thanks!