Displaying spatial mapping meshes like the 3D Viewer in c++

wzeitler

Hey,
I would like to disply the environment like the hololens web tool in the 3D view.
For that I generate meshes with the SurfaceObserver and send them to a server on my desktop PC, where I can read the send data and display the received meshes.
The problem I currently have is that their positions in the world view are not correct, they fit if I just use one meshCollection (amount of meshes returned from the observer) but if if move with the hololens and have a different position in the world the meshes get display in each other.
The following code shows how I apply the transfrom to the current mesh:

XMMATRIX scaleTransform = XMMatrixScalingFromVector(XMLoadFloat3(&m_surfaceMesh->VertexPositionScale));
XMMATRIX transform;

transform = XMLoadFloat4x4(&transformValue);
transform = scaleTransform * transform;

applyTransform = true;

for (int index = 0; index < vertexCount; ++index)
{
    // Read the current position as an XMSHORTN4
    XMSHORTN4 currentPosition = XMSHORTN4(rawVertexData[index]);
    XMFLOAT4 vals;

    // XMVECTOR know hot to convert XMSHORTN4 to actual floating point coordinates
    XMVECTOR vec = XMLoadShortN4(&currentPosition);

    // Store that into an XMFLOAT4 so we can read the values;
    XMStoreFloat4(&vals, vec);

    XMFLOAT4 scaledPos;

    // Scale by the vertex scale
    if (!applyTransform)
    {
        scaledPos = XMFLOAT4(vals.x * vertexScale.x, vals.y *vertexScale.y, vals.z * vertexScale.z, vals.w);
    }
    else
    {
        XMVECTOR scaledVec = XMVector4Transform(vec, transform);
        XMStoreFloat4(&scaledPos, scaledVec);
    }

The transformValue is generated like this:

SpatialCoordinateSystem^ currentCoordinateSystem = m_referenceFrame->GetStationaryCoordinateSystemAtTimestamp(prediction->Timestamp);
auto transformToOrigin = currentCoordinateSystem->TryGetTransformTo(m_stationaryReferenceFrame->CoordinateSystem);
Windows::Foundation::Numerics::float4x4 transformValue = transformToOrigin->Value;

The m_stationaryReferenceFrame->CoordinateSystem is the coordinateSystem at the start of the application and I use it as my reference point for all meshes.

Has anybody a suggestion what I should change or what I do wrong?