The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Can Hololens provide mixed reality magnification solutions for low vision users?
'm a low vision person. My retinas are mostly destroyed due to a birth defect. Although I can see, my eyes do not magnify normally. Thus, I am very nearsighted. I have to be close to everything. Whereas a person with normal vision reads at 18 inches, I have to be about 2 inches away to read it .. that is if the print isn't too small. Or, sitting in class reading a chalk board ... can't do it. Sitting in meeting at work trying to read white board. Again, can't do it. At a normal distance of say 3-4 feet, the writing is too small because my eyes just don't magnify like everyone else's.
My interest in Hololens is to have a camera in the headset and have the camera zoom things up for me. I envision a window being presented to me (hopefully a big window) which I can position close to my face. The camera would be looking at say a regular white board 3-4 or maybe 6-8 (etc) feet away. But the holographic window that Hololens would display would be right in front of my face, and I could zoom the contents of that window to magnify it as well. Then I can get the virtual screen as close as i need it and zoom what's on the virtual board so I can read it. Then in theory, I should be able to attend a class and sit at a normal distance, and have what is on the white board (which is physically at say 3-4 feet away - which does not work for me) displayed right in front of my face .. in a window ... on the Hololens.
In other words, Hololens would make it possible to have the whiteboard right in front of my face, allow me to zoom the image on the virtual whiteboard up and down, so that I can tell what is written on it. Therefore, I could be successful in such a class. Without it, I can see there is writing on the physical board, but it just looks like a bunch of black writing. I can't make it out, and I have to listen to the presenter, and just lisetning does NOT always get the job done. You have to be able to see what he's talking about in order to understand it.
Another application would be to be sitting at my piano, have the Hololens look at the music, capture the actual music at say 2 feet away, but present a virtual window which would be closer, where can zoom up the music on the virtual screen. This would be useful because I can't read the music ... tell what the notes are ... if it's sitting a normal distance away at a normal size.
Hopefully you understand what I'm asking for. Can anybody tell me if what I'm wanting is even possible? One problem that existing low vision solutions have is that you put their headgear on and it gives you this small window to look in. In their window, you see a magnified image, but the window is so small that you can't get very much in the window. If the Holonens can holographically present a big screen in front of you, and if you can zoom the contents up, then in theory you should have an entire TV screen in front of you, rather than just a small window. This sould allow you to see the whole whiteboard, or the whole sheet of music, etc. Seeing the "whole page" would really improve your ability to understand the big picture.
Hope this makes sense. Any responses would be appreciated.
Mike
Answers
I have been thinking about something along the same lines. One aspect would be to magnify what you are reading for the sight impaired, the other would be to do a voice to text translation so the hearing impaired would get captions of a conversation.
However, HoloLens does have some limitations to achieve both of these this. The first being the forward facing camera isn't that higher resolution - but it might be good enough. It would have to be one of those things that would need to be tried out!
The microphone and the Hololens CPU processing probably wouldn't be to doing voice or music processing. But that could be worked around by having another device with a good microphone and share the results with the user via a web service.
Accessibility tools using HoloLens will be in great demand. But your remark about camera resolution is significant. Theoretically, a separate 4K or higher camera device (handheld, smartphone or lapel pin or something) could be controlled through gestures to capture a zone with optical zoom, then throw that image into a window virtually attached to a nearby wall or hanging in space. Similarly, a separate directional microphone could send audio to a voice recognition app which in turn sends the transcription to crawl along the display. Both would use HoloLens only as a display overlay for mixed reality benefits. (Now you're going to remind me that HoloLens display resolution is also a bit limited at the moment...)
So feature one is to throw a live smartphone or webcam image up into an overlay window in the HoloLens.
Feature two is to use a bulls eye overlay to aim a directional microphone at speakers and both play that audio through the speakers but also crawl the text as it is recognized.
And, both these could be apps that display through a tablet even before they are integrated with a HoloLens. Hmm. I actually think building this into Skype first, then into HoloLens makes the most sense. Low Vision users already get instant assistance by making Skype calls to caregivers who tell them what they see. No reason this couldn't use speech recognition and zoom and OCR, etc.
Now I'm going to have to find a Skype engineer...
However, eventual HoloLens development would be "hands free" and fully mixed reality...
Thanks for your part in a thread to think through these features...
To CaptainFaris
You mention in your reply ... "a handheld camera" .... that Hololens would use to get the image from, then process it.
Here's something you need to be aware of. When performing tasks such as playing a musical instrument or sitting in a class taking notes (and you want to have Hololens render the music or display the white board) ... you REALLY need to have your hands free to do the activity. You do NOT want to force the user to have to hold a device in their hands because some activities such as what I've named really require the hands to be free, and headmounted camera is where the work takes place. Perhaps in some activities a person could hold a device, but at least in the situations I've outlined, you need your hands free while your head is looking at what needs to be magnified.
Developers NEED to understand the importance of this when thinking about solutions. A good thing in doing accessibility work would be to speak with low vision users and find out how they would use the device. If you get a good user who has been around magnification solutions for awhile (such as myself) they'll be able to tell you what things they need, and might be able to tell you what to avoid. I think by taking a colaborative approach, you stand a better chance of giveing the low vision user what they need. And after all, isn't that the point to the exercise?