Abstract: Recent advances in scanning technologies allow large-scale scanning of urban scenes. Commonly, such acquisition incurs imperfections: large regions are missing, significant variation in sampling density, noise and outliers. Nevertheless, building facades often consist structural patterns and self-similarities of local geometric structures. Their highly structured nature, makes 3D facades amenable to model-based approaches and in particular to grammatical representations. We present an algorithm for reconstruction of 3D polygonal models from scanned urban facades. We cast the problem of 3D facade segmentation as an optimization problem of a sequence of derivation rules with respect to a given grammar. The key idea is to segment scanned facades using a set of specific grammar rules and a dictionary of basic shapes that regularize the problem space while still offering a flexible model. We utilize this segmentation for computing a consistent polygonal representation from extrusions. Our algorithm is evaluated on a set of complex scanned facades that demonstrate the (plausible) reconstruction.