Abstract: Texture-by-Numbers is an attractive texture synthesis framework, because it is able to cope with non-homogeneous texture exemplars, and provides the user with intuitive creative control over the outcome of the synthesis process. Like many other exemplar-based texture synthesis methods, its basic underlying mechanism is neighbourhood matching. In this paper we review a number of commonly used neighbourhood matching acceleration techniques, compare and analyse their performance in the specific context of Texture-by-Numbers (as opposed to ordinary unconstrained texture synthesis). Our study indicates that the standard approaches are not optimally suited for the Texture-by-Numbers framework, often producing visually inferior results compared to searching for the exact $L^2$ nearest neighbour. We then show that performing Texture-by-Number using the Texture Optimization framework in conjunction with an efficient FFT-based search is able to produce good results in reasonable running times and with a minimal memory overhead.