Issues with Hololens(UWP)<->Sphero bluetooth connection

qdrien

TL;DR

---

C# project that is part of a Windows app works but the corresponding dll throw exceptions when I try to use it from a Unity-generated UWP project.

---

Hello,

I'm trying to build a game using the Hololens and a Sphero Ball.
The official Sphero SDK for Windows hasn't been updated for years but some people managed to make the UWP<->Sphero connection work (see this blog article that even shows it works on the Hololens).

The problem is I'm not building a standard Windows 2D app but a game with Unity.
Therefore, I can't replicate what he did in the blog post: I can't just add another project in my solution.
In fact, I can generate a visual studio solution from Unity and add the external project but:

• Each Unity build removes the project's reference (not really an issue, I could simply add that reference every time I build the game)
• Even adding a simple using RobotKit; in the code prevents Unity from building the game, regardless of platform-dependent directives (such as #if !UNITY_EDITOR && UNITY_WSA_10_0).

The standard way of adding such UWP-specific features in a Unity game is to build a dll and use it for that platform only.
Once configured in Unity, the RobotKit dll is correctly referenced and its classes and methods are available from the Unity-generated solution.

However, when I try to run the solution (no need for the Hololens here, it should work on my local machine), an exception is thrown in the dll's code.
Apparently, there is an issue with a call to RfcommDeviceService.FromIdAsync() that throws a System.InvalidOperationException.
From this page I see that:

If the device acting as the RFCOMM server does not have the correct channel registered in its manifest, this method throws an exception when called on the device acting as the RFCOMM client.

As the exact same code (just not built into a dll) runs fine on a WPF solution, I don't really think that it is the issue.

Another thing worth pointing out is that the ball shows with a different name: It's called "Sphero-ORR" on the WPF solution and "AMP-SPP" on the Unity-generated solution.
A piece of code in the dll indicates that this might be caused by a missing permission in the manifest but I already made sure it was properly set (by checking the bluetooth capability at first, then by specifying the service id).
Here is a link to the piece of code that seems to cause trouble.

Any help would be much appreciated, I'm currently about to give up on that direct bluetooth connection and go for an Android app as a relay (already implemented a WebSocket client/server for that purpose but it's slow and ugly...).

HoloSheep

@qdrien
Have you tried wrapping your UWP specific code with

#if WINDOWS_UWP
           //  --- UWP logic goes here
#endif

as described in other threads in this forum and in the Unity docs?

HTH.

qdrien

@HoloSheep
Thanks for replying.
I in fact use similar directives. To be specific, I am wrapping UWP-specific code with:

#if !UNITY_EDITOR && UNITY_WSA_10_0
//code
#endif

(that I also got from Unity docs). Otherwise Unity wouldn't build.

The issues here are:

• I can't simply add a C# project in the Unity-generated solution for reasons mentionned in the first post
• When I generate the dll from that project and use it in my code, running the application throws a System.InvalidOperationException. (the code using the dll's classes and methods is also wrapped with the aforementionned directives to prevent the editor from complaining and refusing to build)

Here is a "minimal (un)working example" to help understand the problem. Note that it uses a dll generated from this project (the project itself works as demonstrated in the blog post mentionned in the 1st post).
The only change I made to the original code before generating the dll is that I replaced those lines by:

Debug.WriteLine("=> Checking device " + item.Name + "(id: "+ item.Id + ")");
Debug.WriteLine("Now trying to call FromIdAsync()");
var spheroService = await RfcommDeviceService.FromIdAsync(item.Id);
Debug.WriteLine("Never reaches that point if bluetooth capabilities are properly set as previous line throws an exception");
Sphero sphero = new Sphero(spheroService);
foundRobots.Add(sphero);
DiscoveredRobotEvent?.Invoke(this, new RobotEventArgs(sphero));

mtaulty

Hi,

I noticed you were referencing my blog post and so I wondered if I might be able to help.

I downloaded the project which you have as a "minimal (un)working example" and when I opened it in Unity and also when I checked the generated Visual Studio project I didn't see a request to use Bluetooth inside of the application manifest.

Is that just an omission or could this be the problem?

I could try and update my blog post to see if I can document how to take the code that I have working on HoloLens in a 2D app across to a 3D app in Unity if that might help - let me know if you think it would and I'm happy to give it a try in the next day or two.

Mike.

qdrien

@mtaulty
Thank you for replying aswell.
Somehow, I forgot to check the bluetooth capability in Unity but I did check it in Visual Studio so that should not be the issue. The .appxmanifest file that is part of the example I uploaded should already contain that capability but I tried to rebuild the application through Unity after fixing that and the issue remains unchanged.

That would in fact be awesome if you could try and make it work because I'm starting to give up after weeks of trying different things to make UWP/Hololens and Sphero communicate directly inside a Unity application. I would certainly be really grateful for it.

Note: In order to make sure I uploaded the right version, I just downloaded it and "YetAnotherTestWithSphero\Build\YetAnotherTestWithSphero\Package.appxmanifest" does seem to contain:

<Capabilities>
<DeviceCapability Name="bluetooth" />
</Capabilities>

Pavel

I faced similar issue on my attempt to connect Hololens as Bluetooth client to phone(android or wp10) as Bluetooth Host via rfcomm. Hololens just doesn't find host, but same app on wp10 working good(with same manifest).
Also PeerFinder API seems to be not working currently on hololens.

Btw switching roles (hololens = host) did the trick but with issues and it's just workaround, the desired approach is hololens = client. So would be nice if someone share his experience/success story in this field.

qdrien

@mtaulty Did you have any luck on connecting to Sphero from a Unity 3D app?