Comparison of Four Subjective Methods for Image Quality Assessment
Rafał K. Mantiuk, Anna Tomaszewska, Radosław Mantiuk
In Computer Graphics Forum, 31(8), 2012.
Abstract: To provide a convincing proof that a new method is better than the state of the art, computer graphics projects are often accompanied by user studies, in which a group of observers rank or rate results of several algorithms. Such user studies, known as subjective image quality assessment experiments, can be very time-consuming and do not guarantee to produce conclusive results. This paper is intended to help design efficient and rigorous quality assessment experiments and emphasise the key aspects of the results analysis. To promote good standards of data analysis, we review the major methods for data analysis, such as establishing confidence intervals, statistical testing and retrospective power analysis. Two methods of visualising ranking results together with the meaningful information about the statistical and practical significance are explored. Finally, we compare four most prominent subjective quality assessment methods: single-stimulus, double-stimulus, forced-choice pairwise comparison and similarity judgements. We conclude that the forced-choice pairwise comparison method results in the smallest measurement variance and thus produces the most accurate results. This method is also the most time-efficient, assuming a moderate number of compared conditions.
Keyword(s): image quality, quality metrics, subjective metrics, ranking, user studies, single stimulus, double stimulus, pairwise comparison, similarity judgements
Article URL: http://dx.doi.org/10.1111/j.1467-8659.2012.03188.x
BibTeX format:
@article{Mantiuk:2012:COF,
  author = {Rafał K. Mantiuk and Anna Tomaszewska and Radosław Mantiuk},
  title = {Comparison of Four Subjective Methods for Image Quality Assessment},
  journal = {Computer Graphics Forum},
  volume = {31},
  number = {8},
  pages = {2478--2491},
  year = {2012},
}
Search for more articles by Rafał K. Mantiuk.
Search for more articles by Anna Tomaszewska.
Search for more articles by Radosław Mantiuk.

Return to the search page.


graphbib: Powered by "bibsql" and "SQLite3."