Abstract: This paper presents a saliency based bag-of-phrases (Saliency-BoP for short) method for image retrieval. It combines saliency detection with visual phrase construction to extract bag-of-phrase features. To achieve this, the method first detects salient regions in images. Then, it constructs visual phrases using the word pairs which are from the same salient regions. Finally, it extracts the bag of visual phrases from the first K salient regions to describe images. Experimental results on Corel 1K and Microsoft Research Cambridge image database demonstrated that the Saliency-BoP method outperforms related methods such as Bag-of-Words (BoW) or Saliency-BoW.