Hello everyone.

The Mixed Reality Forums here are no longer being used or maintained.

There are a few other places we would like to direct you to for support, both from Microsoft and from the community.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.

If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.

And always feel free to hit us up on Twitter @MxdRealityDev.

HoloLens with Autodesk MotionBuilder

BrekelBrekel ✭✭
in Projects

Prototype showing live streaming of motion capture data from Autodesk MotionBuilder on the laptop into the Microsoft HoloLens over WiFi.

https://www.youtube.com/watch?v=yfl7pwXftUs

The idea behind it is to give visual effects supervisors and movie directors the ability to see virtual characters on their movie set while walking around and without the need for looking through a camera/tablet screen.
While at the same time providing them with a toolset for manipulating the virtual assets with software the crew is already familiar with.

Oh and a link to a small article on Road To VR with some more background info:
roadtovr.com/hololens-autodesk-motionbuilder-app-lets-sfx-artists-build-fantasy-in-reality/

Comments

  • holodoctorholodoctor ✭✭

    The video is pretty cool. Thanks for sharing. ~h

  • neerajwadhwaneerajwadhwa mod

    @Brekel absolutely amazing! If you can think of any more components that would be useful in the HoloToolkit please do enumerate them. Would be happy to see your contributions too!

  • Jimbohalo10Jimbohalo10 ✭✭✭

    @Brekel The best demo of HoloLens ever seen. This is much better than the demos at //Build 2016, because you have used the basic HoloLens, no special cameras or tricks.
    You have set the bar for others to aspire.

    The important point is you have adapted commercially available software to show what is possible

    Thank you for showing us what can be done

  • BrekelBrekel ✭✭

    Thanks guys, glad you like it.
    Hopefully I'll have a part 2 to show somewhere in July. (if what I have in my mind actually works) :)

  • BrekelBrekel ✭✭

    @neerajwadhwa one thing that comes to mind would be components for buttons/sliders/switchers. Along the lines of what Leap Motion is doing for example: https://www.youtube.com/watch?v=SJnWoAI6Kuo

  • neerajwadhwaneerajwadhwa mod

    Noted. Out of curiosity, what did you end up using for networking?

  • BrekelBrekel ✭✭

    I tried getting my favorite lib ZeroMQ to work but couldn't get it to work on UWP.
    I ended up writing my own C++ class based on raw winsocks.

  • BrekelBrekel ✭✭

    Some other ideas for HoloToolkit: anything that can speed up the build process (compile/deploy through commandline instead of full VS?) and stuff that can help for profiling.
    I guess that's more for the SDK/Unity devs and not the toolkit though.

  • phockettphockett ✭
    edited July 2016

    Obviously you already know this, but let me also say: this is awesome. I'd love to do something similar for data visualization projects (likely based around Matlab or Python code on the back-end) but have, so far, only tested very basic static data space export to Hololens - you can find my examples via this thread. Aesthetically I'm pretty happy, but the methodology is pretty lame compared to your demo!

  • DanBourqueDanBourque ✭

    @Brekel said:
    I tried getting my favorite lib ZeroMQ to work but couldn't get it to work on UWP.
    I ended up writing my own C++ class based on raw winsocks.

    This is awesome! Fantastic work!

    Is there any chance you would consider sharing your C++ class based on winsocks? I simply need to read and write via web sockets, but haven't been able to get any of the popular packages to deploy on UWP. I don't need wss support, just plain ws.

    Cheers,
    Dan.

  • BrekelBrekel ✭✭

    Sorry but I can't release code at the moment.

  • danbatisdanbatis ✭

    Hi @Brekel , your demo is pretty cool. I also created a server and client in c++. The plan was to have a client in unity c#. The problem is System.Net.Sockets does not work with the Hololens build. So, for your client running in the Hololens, what did you use? Unet? or maybe Windows.​Networking.​Sockets ?

  • BrekelBrekel ✭✭

    Well I couldn't use Unet since my source data wasn't in Unity.
    I couldn't use anything .NET since my source data is sent from a native C++ plugin for MotionBuilder.
    There are several (very good) native C++ libraries supporting pretty much any platform, none have UWP support though (at least that know of).

    So I ended up coding my own C++ class based on good 'ole Winsock and implemented it as a C++ plugin for MotionBuilder and one for Unity.

  • danbatisdanbatis ✭

    Hi @Brekel , I am still struggling to connect with the Hololens. I am using Windows.Networking.Sockets; Debugging everything was going great, then, when I deploy the application, it crashes right after connecting to my c++ server. In visual studio when I debug I have the same behavior, in Debug it works, in Release it crashes after the connection. Any tips?

    using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.VR.WSA.Input;
using UnityEngine.Windows.Speech;
using System.Linq;

#if UNITY_UWP && !UNITY_EDITOR
using System;
using System.IO;
using Windows.Networking.Sockets;
using Windows.Storage.Streams;
using System.Threading.Tasks;
using System.Runtime.InteropServices.WindowsRuntime;
#endif

public class IncursionManagerTest : MonoBehaviour {
    string hostName = "10.1.6.91";
    string serverPort = "20602";

    public Text modeTxt;

    string commtest;

    int cycles;
    float timer;
    public Text fpsTxt;
    public float updateFPS = 0.5f;

#if UNITY_UWP && !UNITY_EDITOR
    StreamSocket socket;
    enum Packet
    {
        P_Test,
        P_ChatMessage
    };
#endif

    private void Start()
    {
        timer = 0.0f;
        #if UNITY_UWP && !UNITY_EDITOR
           Conecta();
        #endif

        Debug.Log("Start finished loggging!!!!");
        print("Start finished print");
    }

    void Update()
    {
        cycles += 1;
        timer += Time.deltaTime;
        if (timer > updateFPS) {
            fpsTxt.text = "FPS: " + Mathf.RoundToInt(cycles / timer).ToString();
            cycles = 0;
            timer = 0.0f;
        }
    }

    void comFin() {
        //To avoid any gameObject access in the async function, we use the string commtest for logging
        modeTxt.text = commtest;
    }

#if UNITY_UWP && !UNITY_EDITOR
    async void Conecta() {

        commtest = "connecting...";

        try
        {
            //Create the StreamSocket and establish a connection to the echo server.
            Windows.Networking.HostName serverHost = new Windows.Networking.HostName(hostName);
            socket = new StreamSocket();
            await socket.ConnectAsync(serverHost, serverPort);
            commtest += " socket connected";

            IBuffer inbufferPacket = new Windows.Storage.Streams.Buffer(1);
            await socket.InputStream.ReadAsync(
                    inbufferPacket, 1, InputStreamOptions.None);
            Packet packType = (Packet)inbufferPacket.GetByte(0);
            string msghexp = BitConverter.ToString(inbufferPacket.ToArray());
            commtest += "\n packet hex: " + msghexp;            
            commtest += " packet type byte: " + BitConverter.ToString(inbufferPacket.ToArray());

            if (packType == Packet.P_ChatMessage) {
                commtest += " packet received ";
            }
            else {
                commtest += " packet problem xx?##$$%";
            }

            IBuffer inbufferMsgSize = new Windows.Storage.Streams.Buffer(4);
            await socket.InputStream.ReadAsync(
                    inbufferMsgSize, 4, InputStreamOptions.None);
            string msghexi = BitConverter.ToString(inbufferMsgSize.ToArray());
            commtest += "\nsize hex: " + msghexi;
            int msgSize = DataReader.FromBuffer(inbufferMsgSize).ReadInt32();
            commtest += " size received " + msgSize.ToString();
            msgSize = 42;

            IBuffer inbuffer = new Windows.Storage.Streams.Buffer((uint)msgSize);
            await socket.InputStream.ReadAsync(
                    inbuffer, (uint)msgSize, InputStreamOptions.None);

            string msghex = BitConverter.ToString(inbuffer.ToArray());
            commtest += "\nmsghex: " + msghex;

            commtest += "\n end of parse ok.";            
        }
        catch (Exception e)
        {
            //Handle exception here. 
            string msgerror = " exception...\n";
            msgerror += e.ToString();
            msgerror += e.Message;

            print(msgerror);
            commtest += msgerror;

            await socket.CancelIOAsync();
        }
        commtest += "\n out of connection block...";
        comFin();
    }

#endif

}
  • danbatisdanbatis ✭

    Hey @Brekel and anyone else that might be interested, I finally succeeded!!

    Well, you are not allowed to make any Unity API calls in async methods, so, in the code above I created a function (comFin) to call after the connection and the parsing, this function only updates my UI text so I can see it is working when deployed.

    Problem was: I was still calling the function inside the async method Conecta, which resulted in a call to a Unity API inside the async.
    Check out the revised version, I set a boolean in the async method, and if this boolean changes, I call the comFin() function in the Update().

    using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.VR.WSA.Input;
using UnityEngine.Windows.Speech;
using System.Linq;

#if UNITY_UWP && !UNITY_EDITOR
using System;
using System.IO;
using Windows.Networking.Sockets;
using Windows.Storage.Streams;
using System.Threading.Tasks;
using System.Runtime.InteropServices.WindowsRuntime;
#endif

public class IncursionManagerTest : MonoBehaviour {
    string hostName = "10.1.6.91";
    string serverPort = "20602";

    public Text modeTxt;

    string commtest;
    bool newConnection;

    int cycles;
    float timer;
    public Text fpsTxt;
    public float updateFPS = 0.5f;

#if UNITY_UWP && !UNITY_EDITOR    
    StreamSocket socket;
    enum Packet
    {
        P_Test,
        P_ChatMessage
    };
#endif

    private void Start()
    {
        timer = 0.0f;
        #if UNITY_UWP && !UNITY_EDITOR
           Conecta();
        #endif

        Debug.Log("Start finished loggging!!!!");
        print("Start finished print");
    }

    void Update()
    {
        cycles += 1;
        timer += Time.deltaTime;
        if (timer > updateFPS) {
            fpsTxt.text = "FPS: " + Mathf.RoundToInt(cycles / timer).ToString();
            cycles = 0;
            timer = 0.0f;
        }

        if (newConnection) {
            comFin();
            newConnection = false;
        }
    }

    void comFin() {
        //To avoid any gameObject access in the async function, we use the string commtest for logging
        modeTxt.text = commtest;
    }

#if UNITY_UWP && !UNITY_EDITOR
    async void Conecta() {

        commtest = "connecting...";

        try
        {
            Windows.Networking.HostName serverHost = new Windows.Networking.HostName(hostName);
            socket = new StreamSocket();
            await socket.ConnectAsync(serverHost, serverPort);
            commtest += " socket connected";

            IBuffer inbufferPacket = new Windows.Storage.Streams.Buffer(1);
            await socket.InputStream.ReadAsync(
                    inbufferPacket, 1, InputStreamOptions.None);
            Packet packType = (Packet)inbufferPacket.GetByte(0);
            string msghexp = BitConverter.ToString(inbufferPacket.ToArray());
            commtest += "\n packet hex: " + msghexp;
            commtest += " packet type byte: " + BitConverter.ToString(inbufferPacket.ToArray());
            if (packType == Packet.P_ChatMessage) {
                commtest += " packet received ";
            }
            else {
                commtest += " packet problem xx?##$$%";
            }

            IBuffer inbufferMsgSize = new Windows.Storage.Streams.Buffer(4);
            await socket.InputStream.ReadAsync(
                    inbufferMsgSize, 4, InputStreamOptions.None);
            string msghexi = BitConverter.ToString(inbufferMsgSize.ToArray());
            commtest += "\nsize hex: " + msghexi;
            int msgSize = DataReader.FromBuffer(inbufferMsgSize).ReadInt32();
            commtest += " size received " + msgSize.ToString();
            msgSize = 42;

            IBuffer inbuffer = new Windows.Storage.Streams.Buffer((uint)msgSize);
            await socket.InputStream.ReadAsync(
                    inbuffer, (uint)msgSize, InputStreamOptions.None);

            string msghex = BitConverter.ToString(inbuffer.ToArray());
            commtest += "\nmsghex: " + msghex;
            /*
            commtest += "separate conversion ";
            string[] hexValuesSplit = msghex.Split('-');
            foreach (String hex in hexValuesSplit)
            {
                // Convert the number expressed in base-16 to an integer.
                int value = Convert.ToInt32(hex, 16);
                // Get the character corresponding to the integral value.
                char charValue = (char)value;
                commtest += charValue.ToString();
            }

            commtest += "\nbytes conversion: ";
            for(int i=0;i<inbuffer.ToArray().Length;i++)
            {
                // Convert the number expressed in base-16 to an integer.
                //int value = Convert.ToInt32(hex, 16);
                // Get the character corresponding to the integral value.
                byte hex = inbuffer.ToArray()[i];
                char charValue = (char)hex;
                commtest += charValue.ToString();
            }
            //} //end of while
            */
            commtest += "\n end of parse ok.";
        }
        catch (Exception e)
        {
            //Handle exception here. 
            string msgerror = " exception...\n";
            msgerror += e.ToString();
            msgerror += e.Message;

            print(msgerror);
            commtest += msgerror;

            await socket.CancelIOAsync();
        }
        commtest += "\n out of connection block...";
        newConnection = true;
    }

#endif

}
  • sbagchisbagchi ✭

    @danbatis said:
    Hey @Brekel and anyone else that might be interested, I finally succeeded!!

    Well, you are not allowed to make any Unity API calls in async methods, so, in the code above I created a function (comFin) to call after the connection and the parsing, this function only updates my UI text so I can see it is working when deployed.

    Problem was: I was still calling the function inside the async method Conecta, which resulted in a call to a Unity API inside the async.
    Check out the revised version, I set a boolean in the async method, and if this boolean changes, I call the comFin() function in the Update().

    Thanks for sharing! I was looking to do something similar.

  • hhhouwthhhouwt ✭

    @danbatis said:
    Hey @Brekel and anyone else that might be interested, I finally succeeded!!

    Well, you are not allowed to make any Unity API calls in async methods, so, in the code above I created a function (comFin) to call after the connection and the parsing, this function only updates my UI text so I can see it is working when deployed.

    Problem was: I was still calling the function inside the async method Conecta, which resulted in a call to a Unity API inside the async.
    Check out the revised version, I set a boolean in the async method, and if this boolean changes, I call the comFin() function in the Update().

    using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.VR.WSA.Input;
using UnityEngine.Windows.Speech;
using System.Linq;

#if UNITY_UWP && !UNITY_EDITOR
using System;
using System.IO;
using Windows.Networking.Sockets;
using Windows.Storage.Streams;
using System.Threading.Tasks;
using System.Runtime.InteropServices.WindowsRuntime;
#endif

public class IncursionManagerTest : MonoBehaviour {
    string hostName = "10.1.6.91";
    string serverPort = "20602";
       
    public Text modeTxt;
    
    string commtest;
    bool newConnection;

    int cycles;
    float timer;
    public Text fpsTxt;
    public float updateFPS = 0.5f;

#if UNITY_UWP && !UNITY_EDITOR    
    StreamSocket socket;
    enum Packet
    {
        P_Test,
        P_ChatMessage
    };
#endif
        
    private void Start()
    {
        timer = 0.0f;
        #if UNITY_UWP && !UNITY_EDITOR
           Conecta();
        #endif
        
        Debug.Log("Start finished loggging!!!!");
        print("Start finished print");
    }

    void Update()
    {
        cycles += 1;
        timer += Time.deltaTime;
        if (timer > updateFPS) {
            fpsTxt.text = "FPS: " + Mathf.RoundToInt(cycles / timer).ToString();
            cycles = 0;
            timer = 0.0f;
        }

        if (newConnection) {
            comFin();
            newConnection = false;
        }
    }

    void comFin() {
        //To avoid any gameObject access in the async function, we use the string commtest for logging
        modeTxt.text = commtest;
    }

#if UNITY_UWP && !UNITY_EDITOR
    async void Conecta() {
        
        commtest = "connecting...";
        
        try
        {
            Windows.Networking.HostName serverHost = new Windows.Networking.HostName(hostName);
            socket = new StreamSocket();
            await socket.ConnectAsync(serverHost, serverPort);
            commtest += " socket connected";
                        
            IBuffer inbufferPacket = new Windows.Storage.Streams.Buffer(1);
            await socket.InputStream.ReadAsync(
                    inbufferPacket, 1, InputStreamOptions.None);
            Packet packType = (Packet)inbufferPacket.GetByte(0);
            string msghexp = BitConverter.ToString(inbufferPacket.ToArray());
            commtest += "\n packet hex: " + msghexp;
            commtest += " packet type byte: " + BitConverter.ToString(inbufferPacket.ToArray());
            if (packType == Packet.P_ChatMessage) {
                commtest += " packet received ";
            }
            else {
                commtest += " packet problem xx?##$$%";
            }

            IBuffer inbufferMsgSize = new Windows.Storage.Streams.Buffer(4);
            await socket.InputStream.ReadAsync(
                    inbufferMsgSize, 4, InputStreamOptions.None);
            string msghexi = BitConverter.ToString(inbufferMsgSize.ToArray());
            commtest += "\nsize hex: " + msghexi;
            int msgSize = DataReader.FromBuffer(inbufferMsgSize).ReadInt32();
            commtest += " size received " + msgSize.ToString();
            msgSize = 42;

            IBuffer inbuffer = new Windows.Storage.Streams.Buffer((uint)msgSize);
            await socket.InputStream.ReadAsync(
                    inbuffer, (uint)msgSize, InputStreamOptions.None);

            string msghex = BitConverter.ToString(inbuffer.ToArray());
            commtest += "\nmsghex: " + msghex;
            /*
            commtest += "separate conversion ";
            string[] hexValuesSplit = msghex.Split('-');
            foreach (String hex in hexValuesSplit)
            {
                // Convert the number expressed in base-16 to an integer.
                int value = Convert.ToInt32(hex, 16);
                // Get the character corresponding to the integral value.
                char charValue = (char)value;
                commtest += charValue.ToString();
            }

            commtest += "\nbytes conversion: ";
            for(int i=0;i<inbuffer.ToArray().Length;i++)
            {
                // Convert the number expressed in base-16 to an integer.
                //int value = Convert.ToInt32(hex, 16);
                // Get the character corresponding to the integral value.
                byte hex = inbuffer.ToArray()[i];
                char charValue = (char)hex;
                commtest += charValue.ToString();
            }
            //} //end of while
            */
            commtest += "\n end of parse ok.";
        }
        catch (Exception e)
        {
            //Handle exception here. 
            string msgerror = " exception...\n";
            msgerror += e.ToString();
            msgerror += e.Message;

            print(msgerror);
            commtest += msgerror;
            
            await socket.CancelIOAsync();
        }
        commtest += "\n out of connection block...";
        newConnection = true;
    }
    
#endif
    
}

    Now, I want to set up a server for my hololens application, and of course the system.Net.Socket is not supported, and I want to know the Windows.Networking.Sockets is ok or not when connecting,sending data or receiving data? Is it a good way for c-s hololens application?
    Hope for your reply! ^-^

Sign In or Register to comment.