Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Relationship between Objective Performance Measurement and 3D Visual Discomfort in Depth Map Upsampling

깊이맵 업샘플링 방법의 객관적 성능 측정과 3D 시각적 피로도의 관계 분석

  • Gil, Jong In (Dept. of Computer and Communications Engineering, Kangwon National University) ;
  • Mahmoudpour, Saeed (Dept. of Computer and Communications Engineering, Kangwon National University) ;
  • Kim, Manbae (Dept. of Computer and Communications Engineering, Kangwon National University)
  • Received : 2013.08.06
  • Accepted : 2013.12.09
  • Published : 2014.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

A depth map is an important component for stereoscopic image generation. Since the depth map acquired from a depth camera has a low resolution, upsamling a low-resolution depth map to a high-resolution one has been studied past decades. Upsampling methods are evaluated by objective evaluation tools such as PSNR, Sharpness Degree, Blur Metric. As well, the subjective quality is compared using virtual views generated by DIBR (depth image based rendering). However, works on the analysis of the relation between depth map upsampling and stereoscopic images are relatively few. In this paper, we investigate the relationship between subjective evaluation of stereoscopic images and objective performance of upsampling methods using cross correlation and linear regression. Experimental results demonstrate that the correlation of edge PSNR and visual fatigue is the highest and the blur metric has lowest correlation. Further, from the linear regression, we found relative weights of objective measurements. Further we introduce a formulae that can estimate 3D performance of conventional or new upsampling methods.

깊이맵은 3D 입체영상의 생성을 위해 중요한 요소이다. 하지만 깊이 카메라를 이용하여 획득한 깊이맵들은 낮은 해상도를 갖는 단점이 있기 때문에 이를 고해상도로 변환하는 연구들이 활발하게 진행되고 있다. 이러한 연구들은 일반적으로 PSNR, Sharpness Degree, Blur Metric 등과 같은 객관적인 평가방법으로 성능을 검증해왔다. 이러한 평가방법 이외에 DIBR로 가상시점(virtual view)을 생성하여 주관적으로 평가하는 연구도 있으나, 입체영상을 생성하여 깊이맵 업샘플링의 성능을 분석하는 것은 많지 않다. 본 논문에서는 다양한 깊이맵 업샘플링 방법들을 이용하여 생성된 입체영상의 주관적 평가와 업샘플링 방법의 객관적 평가 결과의 상관관계 및 선형회귀법을 이용하여 관련성을 분석한다. 실험결과에서는 에지 PSNR이 시각적 피로도와의 상관관계가 가장 높고, Blur Metric은 가장 낮다는 것을 보여준다. 또한 선형회귀에서는 최적의 입체영상을 얻을 수 있는 객관적 평가의 가중치를 구하고, 기존 또는 새로운 업샘플링 알고리즘의 3D성능을 예측할 수 있는 공식을 보여준다.

Keywords

References

  1. H. Hou and H. Andrews, "Cubic splines for image interpolation and digital filtering," IEEE Trans. Acoust. Speech Signal Process., vol. 26, no. 6, pp. 508-517, Dec. 1978. https://doi.org/10.1109/TASSP.1978.1163154
  2. Tomasi, C., and Manduchi, R. "Bilateral filtering for gray and color images," In Proc. IEEE Int. Conf. on Computer Vision, pp. 836-846, 1998.
  3. J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, "Joint Bilateral Upsampling," ACM Trans. on Graphics, vol. 26, no. 3, 2007.
  4. C. Pham, S. Ha, and J. Jeon, "A local variance-based bilateral filtering for artifact-free detail- and edge-preserving smoothing," PSIVT, Part II, LNCS 7088, pp. 60-70, 2011.
  5. S. Jang, D. Lee, S. Kim, H. Choi, M. Kim, "Depth Map Upsampling with Improved Sharpness," Jounal of Broadcast Engineering, Vol. 17, No. 6, pp. 933-944, Nov. 2012. https://doi.org/10.5909/JBE.2012.17.6.933
  6. Q. Thu and M. Ghanbari, "Scope of validity of PSNR in image/video quality assessment," Electronics Letters, vol. 44, no.13, pp. 800-801, Jun. 2008. https://doi.org/10.1049/el:20080522
  7. C. Tsai, H. Liu, M. Tasi, "Design of a scan converter using the cubic convolution interpolation with canny edge detection," 2011 International Conference on Electric Information and Control Engineering (ICEICE), pp. 5813-5816, Apr. 2011.
  8. P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, "Perceptual blur and ringing metrics: application to JPEG2000," Int. Workshop Multimedia Signal Processing, pp. 403-408, Oct. 2008.
  9. A. Redert, M. Op, C. Fehn, W. IJsselsteijn, M. Pollefeys, L. Van Gool, E. Ofek, I. Sexton, P. Surman, "ATTEST-Advanced Three-dimensional Television System Techniques," Proc. of 3DPVT, pp. 313-319, 2002.
  10. C. Fehn, "Depth-image-based Rendering (DIBR), Compression and Transmission for a New Approach on 3D TV," Proc. of SPIE Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93- 104, 2004.
  11. L. Zhang, W. J. Tam, "Stereoscopic Image Generation Based on Depth Images for 3D TV," IEEE Trans. on Broadcasting, vol. 51, pp. 191-199, 2005. https://doi.org/10.1109/TBC.2005.846190
  12. G. Borgefors, "Hierarchical chamfer matching: a parametric edge matching algorithm," IEEE T. Patten Anal. Mach. Intell., vol. 10, no. 6, pp.849-865, 1988. https://doi.org/10.1109/34.9107
  13. "Subjective assessment of stereoscopic television picture", ITU-R Recommendation BT. 1438, 2000.
  14. G. Nur, S. Dogan, H. Kodikara Arachchi, and A. M. Kondoz, "Impact of Depth Map Spatial Resolution on 3D Video Quality and Depth Perception," 3DTV-CON, pp, 1-4, 2010.
  15. Chang, Kang-Tsung. "Computation for Bilinear Interpolation" Introduction to Geographic Information Systems 5th ed, 2009.
  16. R. Keys, "Cubic convolution interpolation for digital image processing," IEEE Trans on Signal Processing, Acoustics, Speech, and Signal Processing 29 (6), pp. 1153-1160, 1981. https://doi.org/10.1109/TASSP.1981.1163711
  17. C. Tomasi, R. Manduchi, "Bilateral filtering for gray and color images," In Proc. IEEE Int. Conf. on Computer Vision, pp. 836-846, 1998.
  18. J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, "Joint Bilateral Upsampling," ACM Trans. on Graphics, vol. 26, no. 3, 2007.
  19. C. Phan, S. Ha, J. Jeon, "A local variance-based bilateral filtering for artifact-free detail and edge preserving smoothing," PSIVT, Part II, LNCS 7088, pp. 60-70, 2011.
  20. D. Yeo, E. Hap, J. Kim, M. Baig, H. Shin, "Adaptive Bilateral Filtering for Noise Removal in Depth Upsampling," SoC Design Conf., pp. 36-39, 2010.
  21. I. E. Sobel, "Camera Models and Machine Perception," Ph.D. dissertation, Stanford University, Palo Alto, Calif., 1970.