The Mixed Reality Forums here are no longer being used or maintained.
There are a few other places we would like to direct you to for support, both from Microsoft and from the community.
The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.
For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality.
If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers, or in our Microsoft Tech Communities forums at https://techcommunity.microsoft.com/t5/mixed-reality/ct-p/MicrosoftMixedReality.
And always feel free to hit us up on Twitter @MxdRealityDev.
Augmented Reality with 3-D Object Measurement
We have extracted the vertex points from the spatial mapping data of our office floor and exported them to our own software (middleware) for point cloud processing:
We have found that the structure of point cloud is based on an 8x8x8-tree.
- 1 block = 8' x 8' x 8' = 8 brick x 8 brick x 8 brick
- 1 brick = 1' x 1' x 1' = 8 voxel x 8 voxel x 8 voxel
- 1 voxel = 1/8' x 1/8' x 1/8'.
What we will be doing is to display a pin-up poster onto a virtual screen in a cylindrical room corner as below:
For this purpose, we have to know accurately the radius, length, position, and orientation of the real object cylinder. The results of the cylinder fitting are shown below:
The next steps are:
1. Impelementation of the procedure to HoloLens
2. Mapping & displaying a pin-up poster onto the virtual cylindrical screen
3. Rendering the wireframe cylinder, if neccessary.
A gazing point of HoloLens will the pin-up position.
We use DirectX.
Joon
Comments
CurvSurf develops and distributes real-time 3D object/geometry recognition, measurement, and extraction SDK for 3D imaging/measurement applications for tablets, mobile and 3D scanning devices.
Applications include Reverse Engineering, AR (Augmented Reality), Robot Vision, Logistic Managements, Deep Learning, Collision Avoidance, UAV, Autonomous Car, and any 3D object recognition applications which needs “Human-Like” robust perception.
CurvSurf will not develop the end-user applications, but, will help clients as software developer do the job of application development specialized in their field.
What CurvSurf will be doing with HoloLens is to demonstrate the functionality of the CurvSurf middleware:
As a concrete application demonstration, CurvSurf will be showing a pin-up poster (Ad-poster) on a real object cylinder with a minimum diameter of 1'.
URL's:
Google +CurvSurf
YouTube Channel CurvSurf
curvsurf.com
PS: Beginning 2017, the CurvSurf SDK will be prepared for public distribution.
Right, HoloLens has to offer a better spatial mapping for what we gaze.
We should be able to precisely scan what is in front of the user on demand.
Advantage of Google Project Tango providing the access to raw depth data.
In next days, CurvSurf will post a comparable video by using HoloLens on the YouTube Channel 'CurvSurf'. You can compare between with/without access to raw depth data.
HoloLens Spatial Mapping Points
https://youtube.com/watch?v=y9v0qEIB7V4
We have extracted vertex points from the spatial mapping of our office.
The point density seems to be too low for retrieving the geometric information of small objects. We expect we could recognize & measure the gym ball.
Shared from my #HoloLens.
Sphere fitting to HoloLens spatial mapping points.
We notice:
1. Motion tracking is highly solid.
2. Spatial mapping truncates convex/concave details.
3. The minimum size of extractable object is about 0.5 m.
4. MS 'must' provide developers with the access to raw depth data.
5. The interpupillary distance (IPD) must be measured automatically.
Photos & videos can be mapped (displayed) onto the real object surface:
1.The world earth map can be displayed onto the real gym ball surface and we can simulate the worldwide airplane flights.
2. A virtual ad-poster can be displayed onto real cylidrical wall.
We use DirectX not Unity.
HoloLens Spatial Mapping Points with Cylinder Extraction
Shared from my #HoloLens.
Cylinder fitting to the HoloLens spatial mapping points of the cylindrical wall-corner of our office.
Because the radius, postion & pose of the real cylindrical wall are known, photos & videos can be mapped (displayed) onto the real object surface: A virtual ad-poster or video can be displayed onto the real cylidrical wall.
The next work steps:
1. Complete the object surface list: Plane, box, sphere, cylinder, cone, and torus
2. Automate the recognition & determination of the surface type.
Is there a way to see the measurements (values in metric and imperial) effectively projected onto any of the graphics? I don't see any measurements in these videos or am I missing something? Thanks!
@Sealeo
The measurements are displayed implicitly per overlaying the virtual objects onto the real objects.
In order to ovelay a virtual object onto the target real object, we need the minimal set of accurate geometric parameter values of the real object:
Our middleware FindSurface do the job of real-time detection, recognition, and measurement of the size, position, and orientation of real objects.
If the input point data are dense and accurate enough, also the type of real objects of plane, box, sphere, cylinder, cone, and torus can be determined automatically.
1Q2017, the FindSurface SDK will be prepared for public distribution and open the new age of '3D Augmented Reality' (Current augmented reality technologies are '2D Augmented Reality' based on image processing and plane extraction). The applications of FindSurface include Augmented Reality, Robotics, Building Information Modeling, Logistics, etc. CurvSurf will not develop end-user applications, but help clients do their job of application development.
CurvSurf also will be preparing a 3D data processing service. Our clients including robots send CurvSurf via wireless communication ways 3D point clouds and images captured by laser scanner, handheld scanner, smartphone, smartglass, smartwatch, or tablet, etc. Then the data processing system of CurvSurf processes the point clouds and images, and, sends the geometric information extracted from the data back to the clients.
CurvSurf's middleware (DLL) for real-time 3D object/geometry recognition, measurement, and extraction has a size of ca. 200kB binary. If the density and the accuracy of 3D measurements are good enough, the middleware works much well.
CurvSurf's middleware is single thread, non multi-processing yet.
It will be distributed publically 1Q2017.
An application field of CurvSurf's middleware FindSurface: Logistics.
https://www.youtube.com/watch?v=cMmln8SujMg
CurvSurf's Logistics Application 2016, 3D object recognition, FindSurface
HoloLens can enter the hall of logistics only after it provides not only spatial mapping but also raw depth data to developers.
There are many other potential application fields: AR, BIM, Robotics, Game, ...
And, there are many other 3-D cameras providing RGBD data in real-time.
CurvSurf is wondering if Microsoft is willing to offer developers the access to raw depth data stream.
This looks very promising. We are actively looking for a solution/partner to create 3D maps of walls, doors, ceiling and floor of a large space like an office building or airport. Please message me.
@claforte
I recommand you to take first a look at the 3-D industry:
Laser Scanning Forum
http://www.laserscanningforum.com/forum
SPAR3D
http://www.spar3d.com/
InterGEO
http://www.intergeo.de
Google +CurvSurf
https://plus.google.com/+CurvSurf
YouTube Channel CurvSurf
https://www.youtube.com/user/finger6207
curvsurf.com
http://www.curvsurf.com/
We CurvSurf are exhibiting next week.
https://youtube.com/watch?v=4RYMR8i8suU
IROS 2016 (Robotics)
http://www.iros2016.org/exhibitors.html
http://www.iros2016.org/industry_forum.html
InterGeo 2016 (Geospatial)
http://intergeo.by-qb.com/list/?fairID=430884&l=en&exhibitorID=506529&l=en
Presentation at Industry Forum of IROS 2016.
iros2016.org/industry_forum.html
3-D Data Processing for Robotic Applications
https://www.youtube.com/watch?v=3QOe5wyK-Uc
Object detection, recognition, and measurement by voice command.
https://youtube.com/watch?v=4RYMR8i8suU
Real-Time Geometric Feature Extraction with 3-D Measuring Device
https://plus.google.com/+CurvSurf/posts/6SkiWaUNeii
Seeing is not yet understanding.
A camera captures a scene, without understanding the contents.
A 3-D printer replicates an object, without understanding the structure.
Microsoft HoloLens, Google Tango, Intel RealSense are all 3-D camera capturing 3-D space, knowing own position and pose in 3-D space, withot understanding the 3-D space captured yet.
How understanding and interpreting the 3-D space captured is the next question.
"Industrial 3-D Augmented Reality"
https://youtube.com/watch?v=ubRWmkN5CnQ
Smart Manufacturing Technology Conference 2016, COEX, Seoul, Korea, 12/15/2016.
Very impressive and solid technology. I personally believe your technology also serves the critical component when doing the object extraction application. Combining with the image recognition technology powered by ML, it could have tremendous value on consumer facing scenarios.
The beginning of wisdom is the admission of one's ignorance.
Taken by HoloLens in a steel coil factory.
https://www.youtube.com/watch?v=_EMoouFBQjE
HoloLens: Steel Coil Measurement
Taken by Google Tango in a steel coil factory.
https://youtube.com/watch?v=_QxiurWNVjc
Google Tango: Steel Coil Measurement
An article on CurvSurf's FindSurface SDK.
"FindSurface: Real-Time, Automatic 3D Object Analysis for Mobile and AR Devices"
http://www.spar3d.com/sponsored/sponsored-software/findsurface-makes-real-time-automatic-3d-object-recognition-reality/
See also: YouTube Channel CurvSurf
https://www.youtube.com/c/CurvSurf
Joon
CurvSurf's FindSurface SDK incl. documentation is available:
https://developers.curvsurf.com/
Developer registration, SDK download, and license activation are possible per online.
Supported OS: Windows Classic.
Another article on CurvSurf's FindSurface SDK.
Speedy, Efficient 3D Data Processing for any 3D Capture Solution
...
That means companies like CurvSurf will be crucial to the future of 3D. The South Korean company has developed an algorithm that can recognize objects in 3D point clouds. What’s more, it’s so fast and efficient that it works real-time with a live sensor feed, and can run on an AR device.
UPCOMING EVENT:
Augmented World Expo 2017
The largest AR & VR event in the world
Santa Clara Convention Center, Califonia
May 31 - June 2, 2017
http://www.augmentedworldexpo.com
Booth No. 543
Showcase product: FindSurface SDK for Windows classic.
CurvSurf is planning to provide live demonstrations of 3-D augmented reality with Google Tango, Intel RealSense, and Microsoft HoloLens.
3-D Augmented Reality
Real-time 3-D Augmented Reality demonstrated at AWE USA 2017, Booth #543 CurvSurf.
https://www.youtube.com/watch?v=_Cao7bVZAMg
This video is taken by Microsoft HoloLens and CurvSurf FindSurface, and edited in 3 cuts:
Cut #1: Finding and measuring sphere, cylinder, and plane in vertex points of spatial mapping with HoloLens.
- HoloLens provides spatial mapping and motion tracking information.
- CurvSurf FindSurface detects, recognizes, and measures the object size, position, and orientation from 3-D point cloud.
https://developers.curvsurf.com/docu.jsp
With the geometric information of real objects, we can make the following simple 3-D AR examples as in Cut #2 & #3.
Cut #2:
- Virtual flying ball around real gym ball with occlusion
- Virtual drawing on real doll house
- Virtual tiling on real floor.
Cut #3: Virtual rotating poster on real column.
We differentiate 3-D AR from current 2-D AR:
- 2-D AR: based on image processing and plane detection
- 3-D AR: based on 3-D object measurement of surface type, size, position, and orientation of real objects.
Joon
3-D Augmented Reality - Google Tango
Sphere fitting determines the position and size of real gym ball with occlusion.
@CurvSurf I noticed your software is mainly extracting simple objects at the moment like balls, pipes, crates etc. Currently at its current stage of development can it do more complex objects (example toaster) or can custom shapes be recognized/created?
@AvionicHologram
Ya.
CurvSurf FindSurface is a middleware for detecting, recognizing, and measuring the shape, size, position, and pose of objects from point cloud. Object shapes include plane, box, sphere, cylinder, cone, and torus.
https://developers.curvsurf.com/
Most of man-made objects, e.g. room, house, building, table, factory, plant, etc. are geometrically composed of simple primitives such as plane, sphere, cylinder, cone, and torus.
A typical table is composed of a plane and 3~4 rods. If the size of plane is about the human hip, it's a stool.
We have demonstrated how CurvSurf FindSurface works with Google Tango, Intel RealSense, Microsoft HoloLens for Real-time 3-D data streaming.
https://www.youtube.com/c/CurvSurf
https://developers.curvsurf.com/docu.jsp
In next months, CurvSurf will reveal:
A video of demonstrative AR application to Apple ARKit. As far as we have found, the point cloud from Apple ARKit using the latest iPhone/iPad without 3-D camera option is highly noisy. Anyway, CurvSurf FindSurface may extract geometric information from that such bad point cloud.
An Autodesk Revit Plugin source code in C# for automatic feature extraction from point cloud by using CurvSurf FindSurface.
A real-time tracking & occlusion C/C++ code for Intel RealSense ZR300.
Joon