Transforming 2D photos into 3D models

Here is an interesting post from ComputerWorld Blogs.
I had read about Microsoft’s version of the same type of service a few weeks ago. We have already started planning how this technology can be integrated into 3D Walkthroughs, 3D Renderings, and 3D Floor Plans to create an even more realistic final product.

By Ian Lamont on Tue, 04/24/2007 – 3:02pm
Courtesy of StumbleUpon, I happened across an interesting service called Fotowoosh that purports to turn 2D photographs into 3D models. The demo is very cool — check out the European street scene with the church and the tree, and you’ll see what I mean.

Further, if this technology (or one like it) can be integrated with other 3D modelling tools, the impact on a number of industries will be profound. Imagine a FPS game based on actual street scenes, a la Escape From New York. Or a virtual world based on real photographs from a specific time or place. Or a Web-accessible 3D real-estate demo created from standard digital photographs and a floor plan.

The Fotowoosh technology works by some sort of training scheme, according to the “About” page:

Mathematically, it is completely impossible to reconstruct a 3D scene from a single image. And yet when we humans look at a photograph, we see not just a plane filled with color and texture, but the world behind the image. How do we do it?

We believe that this amazing ability of humans comes from years of experience of living in a highly structured world, in which most scenes consist of vertical objects resting on a ground plane. Our insight is that if we can just figure which parts of the image correspond to ground, vertical surfaces, and the sky, we can often construct a simple 3D model of the scene. Our approach is to learn the structure of the world and the appearance of geometric surfaces from a large set of training images. We can then apply that knowledge to new photographs. If we can determine where the vertical surfaces contact the ground in the image, we can recover the depth of those surfaces (up to a scale), giving us a 3D model.

To create the final result, we simply texture map from the original image onto the model.

TechCrunch reports that Fotowoosh uses VRML, and also notes that Microsoft is working on a similar technology called Photosynth. However, when I tried the Photosynth demo (warning: bad installation and clunky UI) I was unable to discern any 3D effects in the images used. Rather, it looked like a way to navigate around a 2D environment pasted onto a set of coordinates existing in a 3D space.