Abstract: We present a novel highlight spots removal algorithm which can remove the highlight spots on the metal surface fast and efficiently. First, multi-light-sources is adopted to reduce the large region of the highlight spots which use several different light sources to get images and use the gradient to reconstruct an image. Multi-light-sources method cannot deal with the overlapped highlight spots. So we use the total variation inpainting model to restore the diffuse region from neighborhood of the specular spots to eliminate highlight spots completely. The light spots on the surface of the metal objects with complex texture can be removed effectively, and the physical property of the surface maintains well. Experiments show that the stereo matching algorithm whose input is the image pair after using the highlight spots removal algorithm can get much better result.