Abstract: Image-based approaches to sketch recognition typically cast sketch recognition as a machine learning problem. In systems that adopt image-based recognition, the collected ink is generally fed through a standard three stage pipeline consisting of the feature extraction, learning and classification steps. Although these approaches make regular use of machine learning, existing work falls short of presenting a proper treatment of important issues such as feature extraction, feature selection, feature combination, and classifier fusion. In this paper, we show that all these issues are significant factors, which substantially affect the ultimate performance of a sketch recognition engine. We support our case by experimental results obtained from two databases using representative sets of feature extraction, feature selection, classification, and classifier combination methods. We present the pros and cons of various choices that can be made while building sketch recognizers and discuss their trade-offs.