Abstract: Human perceptual expertise has significant influence on medical image inspection. However, little is known regarding whether experts differ in their cognitive processing or what effective visual strategies they employ for examining medical images. To remedy this, we conduct an eye tracking experiment and collect both eye movement and verbal description data from three groups of subjects with different medical training levels. Each subject examines and describes 42 photographic dermatological images. We then develop a hierarchical probabilistic framework to extract the common and unique eye movement patterns exhibited among multiple subjects' fixation and saccadic eye movements within each expertise-specific group. Furthermore, experts' annotations of thought units on the transcribed verbal descriptions are time-aligned with these eye movement patterns to identify their semantic meanings. In this work, we are able to uncover the manner in which these subjects alternated their viewing strategies over the course of inspection, and additionally extract their perceptual expertise so that it can be used for advanced medical image understanding.