Augmented Reality with 3-D Object Measurement

CurvSurf · August 2017

The point cloud generated by Apple ARKit with iPhone/iPad without 3-D camera is highly noisy.

However, one of the advantages of the current combination of ARKit with iPhone/iPad is that the measurement distance is limitless if you can translate large enough your iPhone/iPad.

You could even measure the moon by using an iPhone/iPad IF you could translate (i.e. without rotating) enough your iPhone/iPad.

The passive 3-D measuring method (photogrammetry, stereo vision, multi-view geometry) will govern in 5 years. The active method (laser scanner) will disappear.

CurvSurf · August 2017

In next few weeks, by using an iPad with ARKit, we will let to fly a virtual airplane around a dome in an amuse park in Korea. The virtual airplane will be occluded when it is flying behind the dome. In order to get a point cloud of the dome, we will need only to move the iPad about 5 m laterally across to the dome. Intel RealSense, Google Tango, Microsoft HoloLens cannot, because the object distance and size is too large.

CurvSurf · August 2017

Today morning I visited the amuse park. The dome was smaller than I guessed. I got a point cloud (the yellow points) by using an iPad and walking around the dome from 5~7 m distance.

In next few weeks, virtual airplane and cars will circulate around the real dome. Of course, with occlusion.

CurvSurf · August 2017

Definitely early Sept. virtual car and airplane will be wandering on and over the real dome in an amuse park in Korea of 37.43373 127.01940.

If they are going or flying behind the dome they will disappear.

I.e., virtual objects are occluded by real objects.

It's a real magic!

CurvSurf · August 2017

We have experimented on ARKit with an iPad Pro.
Even though the point cloud is highly noisy, CurvSurf FindSurface extracted successfully a sphere for gym ball and a plane for floor. The extracted sphere and plane are in contact.

Gym ball

Point Cloud generated by an iPad Pro with ARKit

Front view

Side view

Top view

CurvSurf · August 2017

Experiment on ARKit with iPad Pro.
https://youtube.com/watch?v=2TlJ_8Tq6js
https://www.youtube.com/watch?v=2TlJ_8Tq6js

CurvSurf · September 2017

CurvSurf will release end Oct. 2017 the C# source code of the Revit Plugin for automatic feature extraction from point cloud by using FindSurface SDK.

The C# source code is free for any usage including modification and distribution.
FindSurface SDK is free for one month testing.

Functions:
• Import point cloud
• Automatic feature extraction of plane, sphere, cylinder, cone, and torus
• Show/hide individual features, inliers, outliers
• Set control parameter values.

Usage:
1. Set program control parameter values:

Point measurement accuracy
Touch radius: Size of point cloud patch
Mean distance: Point cloud density
Threshold of cone’s vertex angle
Region growing: Radial expansion, lateral extension.

Select feature shape to be extracted: Plane, sphere, cylinder, cone, torus, or any
Click a point among the point cloud
Repeat again from 1, 2, or 3.

CurvSurf · December 2017

Real Time 3-D Object Recognition

OS: Windows 10
CPU: Intel Core i7-6500U @ 2.5GHz
Depth Camera: Intel RealSense SR300
Middleware: CurvSurf FindSurface.

CurvSurf FindSurface detects, recognizes, and determines the shape, size, location, and orientation of object surfaces in real time.

The RGB images are not used yet, only point cloud.
FindSurface is working in single-thread mode.

https://youtube.com/watch?v=6tn4rMeIBaw

CurvSurf · December 2017

@CristianGómez said:
Hi @CurvSurf,

My team and I are developing a game in which we need to detect objects in real time. I would like to know if your skd can be easily integrated with HoloLens and Holotoolkit, since we need this library in order to scan.

I look forward to hearing from you soon.

Best regards,
Cristian.

Hi Cristian,

HoloLens is not a device for real time application at all.

HoloLens provides only sparse mesh (spatial mapping, not raw depth data) updated at ca. 1 fps.

FYI,
Google Tango has 5 fps, Intel RealSense has max. 120 fps updating speed of raw dense depth data stream.

As you may know CurvSurf FindSurface can handle raw dense depth data stream in real time.

Joon

CristianGómez · December 2017

We developed a process for scanning in real time with HoloLens but, it is slow so we had to ignore this solution. Because of this, we want to know is CurvSurf FindSurface can help us to detect objects like boxes as well as working with Holotoolkit.

In case that CurvSurf FindSurface was good option for us, what are the steps for integrating it with HoloLens?

Thanks in advance,
Cristian.

CurvSurf · December 2017

@CristianGómez

CurvSurf FindSurface will be available End Jan. 2018 under license. CurvSurf FindSurface is working on Windows (not UWP) and Linux.

YouTube Channel CurvSurf includes videos of detecting and determining the shape, size, position, and orientation of primitive objects including planes, boxes, spheres, cylinders, cones, and tori.

Best,
Joon

CurvSurf · January 2018

Real Time 3-D Object Recognition

OS: Windows 10
CPU: Intel Core i7-6500U @ 2.5GHz
Depth Camera: Intel RealSense SR300
Middleware: CurvSurf FindSurface.

https://www.youtube.com/watch?v=6tn4rMeIBaw

CurvSurf · January 2018

3-D Augmented Reality - Apple ARKit

Virtual airplane rotates around a gym ball with occlusion.
Sphere fitting to point cloud determines the position and size of gym ball.

Used:

Apple iPad Pro
Apple ARKit
Unity
CurvSurf FindSurface.

We will be rotating a virtual real-sized airplane around a real dome of 40 m diameter with occlusion.

Active mobile 3-D cameras, e.g. Google Tango, MS HoloLens, Intel RealSense, are categorically excluded from generating point cloud of large objects. Only passive 3-D measurement methods are under consideration, e.g. ARKit, ARCore, Stereo Vision, etc.

https://youtube.com/watch?v=FzdrxtPQzfA