[go: up one dir, main page]

Performance Comparisons of Subjective Quality Assessment Methods for Video

Toshiko TOMINAGA
Masataka MASUDA
Jun OKAMOTO
Akira TAKAHASHI
Takanori HAYASHI

Publication
IEICE TRANSACTIONS on Communications   Vol.E97-B    No.1    pp.66-75
Publication Date: 2014/01/01
Online ISSN: 1745-1345
DOI: 10.1587/transcom.E97.B.66
Print ISSN: 0916-8516
Type of Manuscript: PAPER
Category: Network
Keyword: 
video quality,  subjective assessment method,  performance comparison,  HD,  QVGA,  H.264,  

Full Text: PDF(2.1MB)>>
Buy this Article


Summary: 
Many subjective assessment methods for video quality are provided by ITU-T and ITU-R recommendations, but the differences among these methods have not been sufficiently studied. We compare five subjective assessment methods using four quantitative performance indices for both HD and QVGA resolution video. We compare the Double-Stimulus Continuous Quality-Scale (DSCQS), Double-Stimulus Impairment Scale (DSIS), Absolute Category Rating method (ACR), and ACR with Hidden Reference (ACR-HR) as common subjective assessment methods for HD and QVGA resolution videos. Furthermore, we added ACR with an 11-grade scale (ACR11) for the HD test and Subjective Assessment of Multimedia Video Quality (SAMVIQ) for the QVGA test for quality scale variations. The performance indices are correlation coefficients, rank correlation coefficients, statistical reliability, and assessment time. For statistical reliability, we propose a performance index for comparing different quality scale tests. The results of the performance comparison showed that the correlation coefficients and rank correlation coefficients of the mean opinion scores between pairs of methods were high for both HD and QVGA tests. As for statistical reliability provided by the proposed index, DSIS of HD and ACR of QVGA outperformed the other methods. Moreover, ACR, ACR-HR, and ACR11 were the most efficient subjective quality assessment methods from the viewpoint of assessment time.