ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Visual-Attention-Aware Progressive RoI Trick Mode Streaming in Interactive Panoramic Video Service

  • Received : 2013.05.29
  • Accepted : 2013.11.19
  • Published : 2014.04.01
Download PDF
⟨ Previous Next ⟩

Abstract

In the near future, traditional narrow and fixed viewpoint video services will be replaced by high-quality panorama video services. This paper proposes a visual-attention-aware progressive region of interest (RoI) trick mode streaming service (VA-PRTS) that prioritizes video data to transmit according to the visual attention and transmits prioritized video data progressively. VA-PRTS enables the receiver to speed up the time to display without degrading the perceptual quality. For the proposed VA-PRTS, this paper defines a cutoff visual attention metric algorithm to determine the quality of the encoded video slice based on the capability of visual attention and the progressive streaming method based on the priority of RoI video data. Compared to conventional methods, VA-PRTS increases the bitrate saving by over 57% and decreases the interactive delay by over 66%, while maintaining a level of perceptual video quality. The experiment results show that the proposed VA-PRTS improves the quality of the viewer experience for interactive panoramic video streaming services. The development results show that the VA-PRTS has highly practical real-field feasibility.

Keywords

References

  1. O. Scheer et al., "Ultrahigh-Resolution Panoramic Imaging for Format-Agnostic Video Production," Proc. IEEE, vol. 101, no. 1, Jan. 2013, pp. 99-114. https://doi.org/10.1109/JPROC.2012.2193850
  2. London Calling Demo Contents, Immersive Media Company, Kelowna, BC, Canada, 1994. Accessed Jan. 15, 2013. http://immersivemedia.com/demos
  3. H.S. Quershi et al., "Quantitative Quality Assessment of Stitched Panoramic Images," IET Image Proc., vol. 6, no. 9, Dec. 2012, pp. 1348-1358. https://doi.org/10.1049/iet-ipr.2011.0641
  4. A. Ahmed et al., "Geometric Correction for Uneven Quadric Projection Surfaces Using Recursive Subdivision of Bezier Patches," ETRI J., vol. 35, no. 6, Dec. 2013, pp. 1115-1125. https://doi.org/10.4218/etrij.13.0112.0597
  5. J. Yoo et al., "Regional Linear Warping for Image Stitching with Dominant Edge Extraction," KSII Trans. Internet Inf. Syst., vol. 7, no. 10, Oct. 2013, pp. 2464-2478. https://doi.org/10.3837/tiis.2013.10.008
  6. H. Kimata, K. Fulazawa, and N. Matsuura, "Partial Delivery Method with Multi-bitrates and Resolutions for Interactive Panoramic Video Streaming System," ICCE IEEE Int. Conf., Las Vegas, NV, USA, vol. 4, no. 1, Jan. 9-12, 2011, pp. 891-892.
  7. M. Makar et al., "Real-Time Video Streaming with Interactive Region of Interest," 17th IEEE Int. Conf. Image Process., Hong Kong, China, Sept. 2010, pp. 4437-4440.
  8. J. Seok et al., "A Visual Perception Based View Navigation Trick Mode in the Panoramic Video Streaming Service," IEICE Trans. Commun., vol. E94-B, no. 12, Dec. 2011, pp. 3631-3634. https://doi.org/10.1587/transcom.E94.B.3631
  9. P. Siebert, T.N.M. Van Caenegem, and M. Wagner, "Analysis and Improvements of Zapping Times in IPTV Systems," IEEE Trans. Broadcast., vol. 55, no. 2, June 2009, pp. 407-418. https://doi.org/10.1109/TBC.2008.2012019
  10. I. Kopilovic and M. Wagner, "A Benchmark for Fast Channel Change in IPTV," IEEE Int. Symp. Broadband Multimedia Syst. Broadcast., Mar. 31 - Apr. 2, 2008, pp. 1-7.
  11. C.Y. Lee, C.K. Hong, and K.Y. Lee, "Reducing Channel Zapping Time in IPTV Based on User's Channel Selection Behaviors," IEEE Trans. Broadcast., vol. 56, no. 3, Sept. 2010, pp. 321-330. https://doi.org/10.1109/TBC.2010.2051494
  12. E. Kurutepe, M.R. Civanlar, and A.M. Tekalp, "Client-Driven Selective Streaming of Multi-view Video for Interactive 3DTV," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 11, Nov. 2007, pp. 1558-1565. https://doi.org/10.1109/TCSVT.2007.903664
  13. G. Muntean, G. Ghinea, and T.N. Sheehan, "Region of Interest-Based Adaptive Multimedia Streaming Scheme," IEEE Trans. Broadcast., vol. 54, no. 2, June 2008, pp. 296-303. https://doi.org/10.1109/TBC.2008.919012
  14. B. Ciubotaru, G. Muntean, and G. Ghinea, "Objective Assessment of Region of Interest-Aware Adaptive Multimedia Streaming Quality," IEEE Trans. Broadcast., vol. 55, no. 2, June 2009, pp. 202-212. https://doi.org/10.1109/TBC.2009.2020448
  15. S. Daly, "Engineering Observations from Spatiovelocity and Spatiotemporal Visual Models," Proc. SPIE Human Vis. Electron. Imag., vol. 3299, Jan. 17, 1998, pp. 180-191.
  16. J. You et al., "Visual Contrast Sensitivity Guided Video Quality Assessment," IEEE Int. Conf. ICME, Melbourne, VIC, Australia, July 9-13, 2012, pp. 824-829.
  17. S. Winkler, "Issue in Vision Modeling for Perceptual Video Quality Assessment," Signal Process., vol. 78, no. 2, Oct. 1999, pp. 231-252. https://doi.org/10.1016/S0165-1684(99)00062-6
  18. Y. Ishiguro and J. Rekimoto, "Peripheral Vision Annotation: Noninterference Information Presentation Method for Mobile Augmented Reality," Proc. 2nd Augmented Human Int. Conf., Tokyo, Japan, Mar. 12, 2011, article no. 8.
  19. R. Szeliski, "Image Alignment and Stitching: A Tutorial," Microsoft Research, Technical Report MSR-TR-2004-92, Oct. 2004. http://research.microsoft.com
  20. Z. Wang et al., "Foveated Wavelet Image Quality Index," Proc. SPIE Appl. Digital Image Process. XXIV, vol. 4472, Aug. 2001, pp. 42-52.
  21. T.M. Bae et al., "Multiple Region-of-Interest Support in Scalable Video Coding," ETRI J., vol. 28, no. 2, Apr. 2006, pp. 239-242. https://doi.org/10.4218/etrij.06.0205.0126
  22. Text of ISO/IEC 14496-10:2005/FDAM 3 Scalable Video Coding, Joint Video Team (JVT) of ISO-IEC MPEG & ITU-T VCEG, Lausanne, N9197, Sept. 2007.
  23. H. Kim et al., "Reducing Channel Capacity for Scalable Video Coding in a Distributed Network," ETRI J., vol. 32, no. 6, Dec. 2010, pp. 863-870. https://doi.org/10.4218/etrij.10.0110.0033
  24. S. Azad, W. Song, and D. Tjondronegoro, "Measuring Bitrate and Quality Trade-off in a Fast Region-of-Interest Based Video Coding," Springer Adv. Multimedia Modeling, vol. 6524, 2011, pp. 442-453.
  25. C. Montgomery et al., Xiph.org Video Test Media (derf's collection), the xiph open source community, 1994. Accessed Aug. 16, 2013. http://media.xiph.org/video/derf
  26. S. Rimac-Drlje, M. Vranjes, and D. Zagar, "Foveated Mean Squared Error - A Novel Video Quality Metric," Multimedia Tools Appl., vol. 49, no. 3, Sept. 2010, pp. 425-445. https://doi.org/10.1007/s11042-009-0442-1
  27. S. Lee, M.S. Pattichis, and A.C. Bovic, "Foveated Video Quality Assessment," IEEE Trans. Multimedia, vol. 4, no. 1, Mar. 2002, pp. 129-132. https://doi.org/10.1109/6046.985561
  28. R. Kooij, K. Ahmed, and K. Brunnstrom, "Perceived Quality of Channel Zapping," Proc. 5th IASTED Int. Conf. Commun. Syst. Netw., Palma de Mallorca, Spain, Aug. 28-30, 2006, pp. 155-158.
  29. S. Lantinga et al., SDL version 1.2.15 open source, Simple DirectMedia Layer forum, 2012. Accessed Jan. 15, 2013. http://www.libsdl.org/download-1.2.php

Cited by

  1. 3차원 기반의 모니터링 시스템과 클라우드 컴퓨팅을 이용한 파노라믹 비디오 서비스 vol.b39, pp.9, 2014, https://doi.org/10.7840/kics.2014.39b.9.590
  2. Quality Assessment of Images Projected Using Multiple Projectors vol.9, pp.6, 2014, https://doi.org/10.3837/tiis.2015.06.015
  3. Fixed Homography-Based Real-Time SW/HW Image Stitching Engine for Motor Vehicles vol.37, pp.6, 2014, https://doi.org/10.4218/etrij.15.0113.1120