Abstract: The result of a scene manipulation is usually displayed by re-rendering the entire image even if the change has affected only a small portion of it. This paper presents a system that efficiently detects and recomputes the exact portion of the image that has changed after an arbitrary manipulation of a scene viewed from a fixed camera. The incremental rendering allows for all visual effects produced by ray tracing, including shadows, reflections, refractions, textures, and bump maps. Two structures are maintained to achieve this. A ray tree is associated with each pixel and is used to detect and rebuild only the modified rays after an optical or geometrical change. A color tree represents the complete color expression of a pixel. All changes affecting the color of a pixel without changing the corresponding ray tree require only re-evaluation of the affected portions of the color tree. Optimizations are presented to efficiently detect the modified structures by the use of strategies such as grouping similar information and building hierarchies. Pruning and weighted re-evaluation of information are also considered to manage the memory requirements. The incremental rendering is done efficiently and accurately and is suitable in an interactive context.