Abstract: We describe an interactive system to reconstruct 3D geometry and extract textures from a set of photographs taken with arbitrary camera parameters. The basic idea is to let the user draw 2D geometry on the images and set constraints using these drawings. Because the input comes directly from the user, he can more easily resolve most of the ambiguities and difficulties traditional computer vision algorithms must deal with.

A set of geometrical linear constraints formulated as a weighted least-squares problem is efficiently solved for the camera parameters, and then for the 3D geometry. Iterations between these two steps lead to improvements on both results. Once a satisfying 3D model is reconstructed, its color textures are extracted by sampling the projected texels in the corresponding images. All the textures associated with a polygon are then fitted to one another, and the corresponding colors are combined according to a set of criteria in order to form a unique texture.

The system produces 3D models and environments more suitable for realistic image synthesis and computer augmented reality.