The best way to generate holograms

Fireman

Hi, so far I have been studying hololens academy tutorials. I found all object (holograms) in the tutorial were so called "assets" that were generated by someone at somewhere else. I can see that generating these assets will become a much bigger issue for holelens to go to mainstream. If you look at the objects around you, table, chairs, TV, computer, cellphone, pens, paper, photoframe, if I want to show them in hololens, what is the most efficient way to generate the "assets" for these objects? Is there a way that we can scan these object to get their 3D information that can be converted to assets?

Thanks,

Fireman

Jarrod1937

The 3D assets are just 3D models imported from various modeling programs. The most common ones are 3DS Max, Maya, and Blender (free). You model your meshes like you would for any video game, then export them to .fbx, then import them into Unity. You have to be aware of things like scale, UV mapping...etc. If you're not familiar with these processes, I would recommend starting with a few beginner video game artwork/modeling articles.

Jarrod1937

The 3D assets are just 3D models imported from various modeling programs. The most common ones are 3DS Max, Maya, and Blender (free). You model your meshes like you would for any video game, then export them to .fbx, then import them into Unity. You have to be aware of things like scale, UV mapping...etc. If you're not familiar with these processes, I would recommend starting with a few beginner video game artwork/modeling articles.

DavidKlineMS

@Fireman,

The assets in the Holographic Academy tutorials were created by 3D and sound artists. Developers, artists and designers are all an essential part of creating a Windows Holographic application.

There are a number of resources for non-artists to acquire assets. For example, Unity has an Asset Store where you can find free and paid models, sounds, shaders and more, that can be used in your applications.

Another excellent source of content is to attend a local meetup and get to know the community. I was at the Seattle hackathon this past weekend and met a large number of excellent artists, designers and developers. I highly recommend joining groups in your area and meeting the great people who are involved in the community.

Thanks!
David

headcasegames

Also you can buy some from turbosquid (or find free ones online somewhere, there are lots of free repos online but much of the time "you will get what you pay for" of course).

And you can try a mobile app like 123D Catch, where you take a series of photos of a subject and can supposedly generate meshes from them. I am actually trying a test with this now, after hearing about it for some time. You will likely still need to do some mesh editing/cleanup on such files, though

Fireman

Thanks all.

But my question is how to generate these assets easily and cheaply without geting a artist involved. I am looking for a way that ordinary people can generate a 3D model for the object around us. For example, one can use his cellphone to scan (taking video) of the object and then upload to a website. Then the data can be accessed and visualized by hololens...

Fireman

@headcasegames 123D catch sounds exactly what I needed. trying it now...

Jarrod1937

@Fireman, if you don't want an artist involved, then you'll want to learn it yourself. While 123D can help, it will still need an artist's touch. Generally photo applications simply can't extract all depth information from a series of photos, so you'll still need to touch up the resulting models. You'll also have to optimize the meshes yourself, as they will not produce game ready meshes. Such tools are a great time saver to an artist, but I haven't seen good video game results straight from such tools.

Jarrod1937

Investigating things further, it looks like, as long as you get some good shots you only have to do a few iterations of conversions and optimization:
https://www.youtube.com/watch?v=-9WpbXshKfw

If you want artist level quality, though with tweaking still needed, checkout this rig:
https://skulltheatre.wordpress.com/2012/09/13/where-the-magic-happens-2/

Fireman

Anyone know if the objects on grabcad.com can be imported to Unity?

LeefromSeattle

Don't forget to check the unity asset store, there are a lot of free models, textures, audio files, animations, example games, etc. Anything you get on the unity asset store can be used commercially.

Stuff randomly found online often doesn't give you rights to use commercially....turbosquid, envato, and a few other stock model sites have some free models as well.

Fireman

@LeefromSeattle I found the assets in Asset Store were design for games and could not meet the quality of industry design. But your comments about license is correct

LeefromSeattle

Well typically cad models and models used in pre rendered production are not suitable for games without heavy modification from an artist. If it's simple it might work but they typically have huge vertice counts, many many meshes and materials (huge number of draw calls) and simply aren't made for realtime rendering.

CurvSurf

@Fireman said:

...
Is there a way that we can scan these object to get their 3D information that can be converted to assets?
...
>

If someone has solved that problem, the ever-problem of computer vision is solved. He/she must have found the Holy Grail of computer vision.

HoloLens delivers meshes of the object surfaces scanned, i.e, 'spatial mapping' generated by the software thechnique of 'KinectFusion'.

The meshes are sufficient for rendering (computer graphics), showing, displaying the object surfaces to you for visually understanding/recognizing the objects.

But, computer vision (not you human) likes to get 3D information of the objects scanned, i.e. the position, orientation, size, shape, etc that meshes cannot deliver directly.

The next years may can answer the question.

Joon

Jarrod1937

@CurvSurf, exactly. Even advanced computer vision just does a highly dense point cloud, itself a rough estimation of the immediate environment. When you're purely looking at the optical component to derive depth you run into quite a few issues where relative material luminosity and specular behavoir can give you false depth queues. This can be overcome with actual depth sensing devices (stereo cameras, LIDAR, depth distorted light patterns...etc), but these are still crude measurements of depth. This is especially true with time of flight devices, where you need a very high sample rate to achieve a few cm of accuracy. Though, I have hope with the new cheap solid state LIDAR's they are coming out with, some interesting possibilities there.

Jimbohalo10

@Fireman Why not use free sources like thingiverse.com Holographic
Hiro AR Tag
Maybe try the search for holograms models

Then import using the STL import/export asset store item https://www.assetstore.unity3d.com/en/#!/content/3397
or maybe just load the STL at runtime using
Runtime STL Loader