Abstract: We present a semi-automatic framework for construction of curved and polygonal 3D models from a 2D line drawing, such as architectural or mechanical drawings. Despite advances in image-based modeling, 3D modeling from a drawn image remains largely manual. In contrast, our method only requires the user to annotate the source image with a drawn cube for camera calibration. 3D models are then generated automatically. The modeling process has four steps: camera calibration, a novel line detection algorithm for noisy input, line labeling to calculate polygon adjacencies, a new incremental construction method that uses plane hinging-angle optimization to improve scalability over previous approaches. We also present algorithms for handling curved surfaces when they are part of a polygonal model that provides boundary conditions.