Abstract: In this paper, we propose a new method, which requires no interactive operation, to estimate background from an image sequence with occluding objects. The images are taken from the same viewpoint under similar illumination conditions. Our method combines the information from input images by selecting the appropriate pixels to construct the background. We have two simple assumptions for the input image sequence: each background pixel has to be disclosed at least once and some parts of the background are never occluded. We propose a cost function that includes a data term and a smoothness term. A unique feature of our data term is that it has not only the stationary term, but also a new predicted term obtained using an image inpainting technique. The smoothness term guarantees that the output is visually smooth so that there is no need for post-processing. The cost is minimized by applying graph cuts optimization. We apply our algorithm to several complex natural scenes as well as to an image sequence with different camera exposure settings, and the results are encouraging.