Abstract: This paper presents a framework and the associated algorithms to perform 3D scene analysis from a single image with lens distortion. Previous work focuses on either making 3D measurements under the assumption of one or more ideal pinhole cameras or correcting the lens distortion up to a projective transformation with no additional metric analyses. In this work, we bridge the gap between these two areas of work by incorporating metric constraints into lens distortion correction to achieve metric calibration. Lens distortion parameters, especially the lens distortion center, can be precisely recovered with this approach. Subsequent 3D measurements can be made from the corrected image to recover scene structures. In addition, we propose an algorithm based on hybrid backward and forward covariance propagation to yield a quantitative analysis of the confidence of the results. Experimental results show that our approach simultaneously performs image correction and 3D scene analysis.