Abstract: High quality labeled data is essential for developing and evaluating sketch recognition algorithms. Unfortunately, labeling freely-drawn sketches is time-consuming and difficult, if not impossible, using current technologies. These difficulties and the resulting lack of labeled data fundamentally limit the development of recognition algorithms.We present an intuitive, direct manipulation pen-based application for labeling sketch data in any two-dimensional domain. Our labeling tool supports the three essential sketch recognition labeling tasks: stroke fragmentation, stroke grouping and label application. Our interface integrates standard and novel interaction techniques to make each task efficient and natural. In a user study, all users felt that labeling data with our tool was quick and efficient.