Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

What does the tracking camera in HP MR headset record?

akshayakshay
edited November 2017 in Q&A and Discussions

Good day everyone,
I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

I was hoping to record video from the two cameras in the headset and I have these doubts -

  • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
  • If its not possible to record video like mentioned above, what is the best alternative to do it?

It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
Thanks in advance!

Best Answers

Answers

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. Windows MR HDMs do not have RGB camera, but they don't have depth sensing cameras either (only the HoloLens do have these ;) ) They do have low quality black and white cameras however. But I'm not even remotely sure if it is possible to get the raw data or if all the tracking calculations are already done within the headset. My guess is that it might be possible to get the raw camera data somehow, but probably not easily.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

  • edited July 2018

    @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • @MAK said:

    @akshay said:
    Good day everyone,
    I have a HP Mixed reality headset with the required system specs as mentioned in https://goo.gl/w35F86 and have installed all the tools to build Windows MR apps (https://goo.gl/yP0HH2).
    I just started working on a project which requires mixed reality and pardon me if I sound naive. I'm able to see the Cliff house in the headset. I'm also able to create simple scenes in Unity and run it on the headset. But now I'd like to work on the real world data and I can't find the API which extracts the real world video from the 2 cameras in the headset.

    I was hoping to record video from the two cameras in the headset and I have these doubts -

    • Do they really record videos or is it just a tracking camera? What type of camera is it and how does it function?
    • If its not possible to record video like mentioned above, what is the best alternative to do it?

    It'd be really helpful if anyone could point me in the right direction to create mixed reality apps.
    Thanks in advance!

    The Windows MR HMDs don't have RGB cameras (only the HoloLens has one) but have 2 Depth Sensing cameras. So you won't be able to "record" anything. The best you could do is extracted the reconstructed mesh that the cams are tracking but this is not possible yet in the APIs (unlike on the HoloLens).

    That is not completely true. They don't have RGB cameras, but they also don't have depth sensing cameras (only the HoloLens do have these ;) ). They do have two low quality black and white cameras, however. If the calculations aren't done within the HMD and discarded immediatly, it could be possible to grab those images. Might still be a difficult task, if possible at all.

    Source: https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system

    The tracking system uses two low-resolution black and white cameras to identify features in your environment in visible light. The system will triangulate its position based on the observed features, and supplement this information by fusing high rate IMU data to produce a continuous pose estimation for the HMD in your environment.

  • edited August 2018

    Oh great... first my comments don't show up or disappear after 5 Seconds, and days (about two weeks) later they get here all at once. :neutral: Sorry for all the spamming.

Sign In or Register to comment.