Abstract: We present a new representation for time-varying image data that allows for varying - and arbitrarily high - spatial and temporal resolutions in different parts of a video sequence. The representation, called "multiresolution video," is based on a sparse, hierarchical encoding of the video data. We describe a number of operations for creating, viewing, and editing multiresolution sequences. These operations support a variety of applications: multiresolution playback, including motion-blurred "fast-forward" and "reverse" constant- speed display; enhanced video scrubbing; and "video clip-art" editing and compositing. The multiresolution representation requires little storage overhead, and the algorithms using the representation are both simple and efficient.