Abstract: In this paper, we propose a novel contour-based volume image segmentation technique. Our technique is based on an implicit surface reconstruction strategy, whereby a signed scalar field is generated from user-specified contours. The key idea is to compute the scalar field in a joint spatial-range domain (i.e., bilateral domain) and resample its values on an image manifold. We introduce a new formulation of Hermite radial basis function (HRBF) interpolation to obtain the scalar field in the bilateral domain. In contrast to previous implicit methods, bilateral HRBF (B-HRBF) generates a segmentation boundary that passes through all contours, fits high-contrast image edges if they exist, and has a smooth shape in blurred areas of images. We also propose an acceleration scheme for computing B-HRBF to support a real-time and intuitive segmentation interface. In our experiments, we achieved high-quality segmentation results for regions of interest with high-contrast edges and blurred boundaries.