Darkroom: compiling high-level image processing code into hardware pipelines
James Hegarty, John Brunhaver, Zachary DeVito, Jonathan Ragan-Kelley, Noy Cohen, Steven Bell, Artem Vasilyev, Mark Horowitz, Pat Hanrahan
In ACM Transactions on Graphics, 33(4), July 2014.
Abstract: Specialized image signal processors (ISPs) exploit the structure of image processing pipelines to minimize memory bandwidth using the architectural pattern of line-buffering, where all intermediate data between each stage is stored in small on-chip buffers. This provides high energy efficiency, allowing long pipelines with tera-op/sec. image processing in battery-powered devices, but traditionally requires painstaking manual design in hardware. Based on this pattern, we present Darkroom, a language and compiler for image processing. The semantics of the Darkroom language allow it to compile programs directly into line-buffered pipelines, with all intermediate values in local line-buffer storage, eliminating unnecessary communication with off-chip DRAM. We formulate the problem of optimally scheduling line-buffered pipelines to minimize buffering as an integer linear program. Finally, given an optimally scheduled pipeline, Darkroom synthesizes hardware descriptions for ASIC or FPGA, or fast CPU code. We evaluate Darkroom implementations of a range of applications, including a camera pipeline, low-level feature detection algorithms, and deblurring. For many applications, we demonstrate gigapixel/sec. performance in under 0.5mm2 of ASIC silicon at 250 mW (simulated on a 45nm foundry process), real-time 1080p/60 video processing using a fraction of the resources of a modern FPGA, and tens of megapixels/sec. of throughput on a quad-core x86 processor.
Article URL: http://dx.doi.org/10.1145/2601097.2601174
BibTeX format:
@article{Hegarty:2014:DCH,
  author = {James Hegarty and John Brunhaver and Zachary DeVito and Jonathan Ragan-Kelley and Noy Cohen and Steven Bell and Artem Vasilyev and Mark Horowitz and Pat Hanrahan},
  title = {Darkroom: compiling high-level image processing code into hardware pipelines},
  journal = {ACM Transactions on Graphics},
  volume = {33},
  number = {4},
  pages = {144:1--144:11},
  month = jul,
  year = {2014},
}
Search for more articles by James Hegarty.
Search for more articles by John Brunhaver.
Search for more articles by Zachary DeVito.
Search for more articles by Jonathan Ragan-Kelley.
Search for more articles by Noy Cohen.
Search for more articles by Steven Bell.
Search for more articles by Artem Vasilyev.
Search for more articles by Mark Horowitz.
Search for more articles by Pat Hanrahan.

Return to the search page.


graphbib: Powered by "bibsql" and "SQLite3."