Reconstructing detailed dynamic face geometry from monocular video
Pablo Garrido, Levi Valgaert, Chenglei Wu, Christian Theobalt
In ACM Transactions on Graphics, 32(6), November 2013.
Abstract: Detailed facial performance geometry can be reconstructed using dense camera and light setups in controlled studios. However, a wide range of important applications cannot employ these approaches, including all movie productions shot from a single principal camera. For post-production, these require dynamic monocular face capture for appearance modification. We present a new method for capturing face geometry from monocular video. Our approach captures detailed, dynamic, spatio-temporally coherent 3D face geometry without the need for markers. It works under uncontrolled lighting, and it successfully reconstructs expressive motion including high-frequency face detail such as folds and laugh lines. After simple manual initialization, the capturing process is fully automatic, which makes it versatile, lightweight and easy-to-deploy. Our approach tracks accurate sparse 2D features between automatically selected key frames to animate a parametric blend shape model, which is further refined in pose, expression and shape by temporally coherent optical flow and photometric stereo. We demonstrate performance capture results for long and complex face sequences captured indoors and outdoors, and we exemplify the relevance of our approach as an enabling technology for model-based face editing in movies and video, such as adding new facial textures, as well as a step towards enabling everyone to do facial performance capture with a single affordable camera.
Article URL: http://dx.doi.org/10.1145/2508363.2508380
BibTeX format:
@article{Garrido:2013:RDD,
  author = {Pablo Garrido and Levi Valgaert and Chenglei Wu and Christian Theobalt},
  title = {Reconstructing detailed dynamic face geometry from monocular video},
  journal = {ACM Transactions on Graphics},
  volume = {32},
  number = {6},
  pages = {158:1--158:10},
  month = nov,
  year = {2013},
}
Search for more articles by Pablo Garrido.
Search for more articles by Levi Valgaert.
Search for more articles by Chenglei Wu.
Search for more articles by Christian Theobalt.

Return to the search page.


graphbib: Powered by "bibsql" and "SQLite3."