Abstract: Depth map generated by the Kinect may have some pixels lost due to echo attenuation of infrared light and mutual interference between neighboring pixels, which can cause pervasive problems when utilizing Kinect cameras as depth sensors. In this work, we propose a 2-step inpainting algorithm to infill the holes. First, a naive Bayesian estimation is conducted as preliminary inpainting scheme, utilizing neighboring pixels of the missing ones, and corresponding pixels in the color image as prior knowledge. After that, an optimization is implemented to improve the depth map, where the false edges in mistakenly inpainted regions are detected, then iteratively propelled to their true positions under total variation framework. Experimental results are included to show effectiveness of the proposed algorithm.