Abstract: Video object cutout aims to extract homogenous objects from background in a video clip, which is a key process in many video processing fields, such as video compositing, video stylized rendering and so on. In this paper, we present a novel video cutout method by matching hierarchical structure of video frames. We first segment each frame by mean shift and construct hierarchical structure as a tree in preprocess stage. We then require user's interaction to label objects in a key frame. We further model video segmentation as matching hierarchical structure of frame and proposed an inter-frame matching algorithm. Experimental results show that our method can achieve desirable video segmentation results.