The visual microphone: passive recovery of sound from video
Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J. Mysore, Frédo Durand, William T. Freeman
In ACM Transactions on Graphics, 33(4), July 2014.
Abstract: When sound hits an object, it causes small vibrations of the object's surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects---a glass of water, a potted plant, a box of tissues, or a bag of chips---into visual microphones. We recover sounds from high-speed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound. We evaluate the quality of recovered sounds using intelligibility and SNR metrics and provide input and recovered audio samples for direct comparison. We also explore how to leverage the rolling shutter in regular consumer cameras to recover audio from standard frame-rate videos, and use the spatial resolution of our method to visualize how sound-related vibrations vary over an object's surface, which we can use to recover the vibration modes of an object.
Article URL: http://dx.doi.org/10.1145/2601097.2601119
BibTeX format:
@article{Davis:2014:TVM,
  author = {Abe Davis and Michael Rubinstein and Neal Wadhwa and Gautham J. Mysore and Frédo Durand and William T. Freeman},
  title = {The visual microphone: passive recovery of sound from video},
  journal = {ACM Transactions on Graphics},
  volume = {33},
  number = {4},
  pages = {79:1--79:10},
  month = jul,
  year = {2014},
}
Search for more articles by Abe Davis.
Search for more articles by Michael Rubinstein.
Search for more articles by Neal Wadhwa.
Search for more articles by Gautham J. Mysore.
Search for more articles by Frédo Durand.
Search for more articles by William T. Freeman.

Return to the search page.


graphbib: Powered by "bibsql" and "SQLite3."