Mapping pixel coordinates to real world objects
I try and try, but I can't figure out, what I am doing wrong. I have to confess, that I am really new to the whole computer vision and holographic thing.
Let me explain what I try to do and then, where my problem is.
I try to build an app for the HoloLens that communicates with a Deep Learning Object Detection Framework. Then after receiving results the HoloLens should transform the pixel coordinates for the found objects, cast a ray in the direction of the found objects and on the hitpoint should build a primitive cube.
I found a project, which seems to be like mine (https://blogs.sap.com/2016/06/09/annotating-the-world-using-microsoft-hololens/) so I used their code for the transformation. I had to build a bit around, because I use my own DL-Server. But the results I get from my server should be the same as their results via Microsoft Cognitive Services(I get top,bot,left,right coordinates from the boundingboxes).
My result cubes are really off from the found objects. I tried to figure it out on my own (I deleted the Scalevector-Function from the other project, because if I understand it right it maps all objects to the size of the holo-screen. But if there are objects found out of the screen I want to map them too.) but I have no clue what I am doing wrong.
I read many threads about, I think, a nearly same problem, but I didn't find any solutions.
I hope I find someone, who could explain me what would be the best thing for me to do.