Moving holograms using gaze and gesture

I've been trying to move holograms using gaze and gesture:

  • airtap object
  • gaze to move object
  • airtap again to place object

I followed the instructions posted earlier on but still could not get it to move.

By @HoloSheep :

After you have correctly copied the HoloToolkit into a new unity project and made all of the appropriate HoloLens project setting adjustments you would then do the following to get the gaze and gesture manager scripts working:

•add empty game object to root of your scene hierarchy (name it ModelBase)
•add 3D model (like a cube) as child of ModelBase
•add 3D box collider as component of ModelBase
•size 3D box collider to slightly larger than 3D model (in all dimensions) be sure to include the models origin inside 3D box collider area
•add Cursor prefab from HTK to root of your scene hierarchy
•add another empty game object to root of your screen hierarchy (name it Managers)
•add GazeManager script from HTK as component of Managers
•add GestureManager script from HTK as component of Managers
•add SpatialMapping prefab from HTK to root of hierarchy (turn off Draw Visual Meshes checkbox)
•add TapToPlace script as component to ModelBase

Build and deploy.
You should be able to gaze at your 3D object and Airtap on it to engage the spatial mapping and move the object around. The next Airtap should place your object where you were gazing at (on the spatial mapped surface) and the spatial mapping should turn off.

I've also made all appropriate setting adjustments.
Appreciate if anyone could help.

Best Answers

Answers

  • @felsiska please describe what you are seeing and what is and isn't working for you.

    I would be happy to try and help work you through it.

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • the cursor is reacting to the cube surface correctly. but nothing happen when I airtap the cube. I've also attached cursor feedback to cursor so that I can see the change of cursor shape when I airtap.

    is there any part of the script that I need to edit?

  • did u check the SpatialPerception? its need to be checked when you use spatial mapping

  • Yes I've checked it. just realized that my GestureManager script is not updated. Its working now after updating them. will explore further.

    Thx for the help

    • Using the TapToPlace script, how do I move the hologram while locking it to the surface mesh to prevent it from "falling over" the mesh?
    • I understand that all objects with TapToPlace script within the same project will move at the same time when manipulation is detected. Any way to only move the object that is tapped at during manipulation?
  • HoloSheepHoloSheep mod
    edited September 2016

    @felsiska

    • not sure I understand what you are referring to when you say "falling over the mesh"

    • The issue of individual single versus multi object response to existing TapToPlace script boils down to the object hierarchy that you place your objects in and the associated colliders. To get multiple object to move independently you could simply repeat the steps of:

    •add empty game object to root of your scene hierarchy (name it ModelBase)
    •add 3D model (like a cube) as child of ModelBase
    •add 3D box collider as component of ModelBase
    •size 3D box collider to slightly larger than 3D model (in all dimensions) be sure to include the models origin inside 3D box collider area

    •add TapToPlace script as component to ModelBase

    and use a unique name for each empty game object (ModelBase1, ModelBase2..) with each having its own child object (3D model)

    Alternatively there are multiple options that are often more complex but involve prefabs and programmatic generation of objects that could also achieve the same results by creating the necessary arrangement of scripts, models and colliders.

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • @HoloSheep

    • regarding issues of objects "falling over the mesh" is similar to how an object go beyond and behind the spatial meshes.
    • Thx for the advice. Managed to move them individually. however it seems that the center of collider is locked to the mesh instead of the corners. this will result in having the object "sinking" below ground level.
    • I've re-used Placeable script from Holograms 230 and managed to lock the object to a surface during manipulation. but object will sink halfway into the surface mesh when placed. how do I change the script so that it will lock to the surface of collider and not the center?
  • @felsiska ah, okay I see. Sounds like you are running into the issue of fixed model origin (aka pivot point).

    See this thread and some of its reference links for a description of the issue and ways to work around it.

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • @HoloSheep

    I am currently only using unity 3d game objects for trial. tried to move and resize the collider so that the center touches the surface of object nicely, but it still sinks below mesh surfaces. is there a way to move the pivot point of the collider? or will it always lock to the center?

    thx so much for helping btw. really appreciate it.

  • understood. its working now. thanks so much for the detailed explanation.

  • currently I have a room model, with furniture, in fbx format. if I want to move the furniture within the room space, will it be better to use the Placeable script instead of TapToPlace? the only problem I have about Placeable is that it will show the collider box instead of just the game object itself while during placing. is it also possible to have the furniture new position to be anchored to the walls/floors of room model instead of the real spatial surfaces?

  • @felsiska said:
    currently I have a room model, with furniture, in fbx format. if I want to move the furniture within the room space, will it be better to use the Placeable script instead of TapToPlace? the only problem I have about Placeable is that it will show the collider box instead of just the game object itself while during placing. is it also possible to have the furniture new position to be anchored to the walls/floors of room model instead of the real spatial surfaces?

    From where do you have the PlaceAble Script? :)

  • @AylinCopoglu,
    I believe @felsiska is using Placeable.cs from the Holograms 230 course, which displays visual queues, in the form of a translucent cube, around the object and performs several raycasts to determine if the object will fit on a surface.

    Thanks!
    David

  • @DavidKlineMS said:
    @AylinCopoglu,
    I believe @felsiska is using Placeable.cs from the Holograms 230 course, which displays visual queues, in the form of a translucent cube, around the object and performs several raycasts to determine if the object will fit on a surface.

    Thanks!
    David

    Thanks David!

  • @DavidKlineMS said:
    @AylinCopoglu,
    I believe @felsiska is using Placeable.cs from the Holograms 230 course, which displays visual queues, in the form of a translucent cube, around the object and performs several raycasts to determine if the object will fit on a surface.

    Thanks!
    David

    Yes, its from Holograms 230. currently the collider size is 2x of the object so that object will sit nicely on the surface. however this will make the object look not to scale during placement where the collider itself will appear and hover across the surface. any ways to hide the collider box and only show the 3d object itself during placement?

  • @DavidKlineMS said:
    @felsiska,
    You can hide the bounding cube by commenting out the following line in the Update method in Placeable.cs:

    DisplayBounds(canBePlaced);

    Thanks!
    David

    thanks so much!

  • In order to move a hologram from one position to another, is it possible to drag (move) it only using gaze - without hand movement. That is,

    • Gaze at the hologram
    • Airtap to select it
    • Move your gaze vector (WITHOUT moving hand)
    • Again airtap at the destination, so as to position the object there.

    Is there any source explaining this?

    Thanks and regards,
    John

  • @John_Smith1

    The Origami demo does exactly that.
    Check out the Holograms 101 tutorial.

    That project allows you to air tap on the holographic pad of paper, gaze around the room and the pad will move to the closest available surface that intersect with your gaze and when you air tap again the origami pad of paper gets repositioned to the new location. (some may object to calling that a "dragging" operation but that is essentially what it is)

    HTH.

    Windows Holographic User Group Redmond

    WinHUGR.org - - - - - - - - - - - - - - - - - - @WinHUGR
    WinHUGR YouTube Channel -- live streamed meetings

  • Thank you for your quick response.

  • Hi, you can try my full tutorial on how to move, resize or rotate objects in your HoloLens apps using MR-Toolkit-Unity 2017.2 version (as latest as it gets) https://codeholo.com/2018/05/06/move-resize-and-rotate-objects-in-your-hololens-apps/

Sign In or Register to comment.