Abstract: A hybrid tracking system, which integrates Computer Vision (CV), GPS and inertial sensing, is presented. It is designed for precise and jitter-free augmentation on feature-rich planar surfaces in outdoor urban environments. Two recently developed CV algorithms, namely keypoint signatures and Efficient Second-order Minimization, are central to achieving the precision and stability required. As CV tracking algorithms are not scalable to large environments and not completely robust, GPS and inertial sensing are used to define a limited search region to initialize CV tracking. The modifications of both CV algorithms for outdoor operations are presented, along with the design considerations for building the proposed hybrid tracker. Experimentation of the tracker and augmentation on real world surfaces are presented.