ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

JPEG Pleno: Providing representation interoperability for holographic applications and devices

  • Schelkens, Peter (Department of Electronics and Informatics, Vrije Universiteit Brussel) ;
  • Ebrahimi, Touradj (Multimedia Signal Processing Group, Ecole Polytechnique Federale de Lausanne) ;
  • Gilles, Antonin (Institute of Research & Technology b<>com) ;
  • Gioia, Patrick (Institute of Research & Technology b<>com) ;
  • Oh, Kwan-Jung (Broadcasting & Media Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Pereira, Fernando (Instituto Superior Tecnico, Universidade de Lisboa-Instituto de Telecomunicacoes) ;
  • Perra, Cristian (Department of Electrical and Electronic Engineering, UdR CNIT, University of Cagliari) ;
  • Pinheiro, Antonio M.G. (Universidade da Beira Interior and Instituto de Telecomunicacoes)
  • Received : 2018.09.11
  • Accepted : 2018.12.10
  • Published : 2019.02.12
Download PDF
⟨ Previous Next ⟩

Abstract

Guaranteeing interoperability between devices and applications is the core role of standards organizations. Since its first JPEG standard in 1992, the Joint Photographic Experts Group (JPEG) has published several image coding standards that have been successful in a plethora of imaging markets. Recently, these markets have become subject to potentially disruptive innovations owing to the rise of new imaging modalities such as light fields, point clouds, and holography. These so-called plenoptic modalities hold the promise of facilitating a more efficient and complete representation of 3D scenes when compared to classic 2D modalities. However, due to the heterogeneity of plenoptic products that will hit the market, serious interoperability concerns have arisen. In this paper, we particularly focus on the holographic modality and outline how the JPEG committee has addressed these tremendous challenges. We discuss the main use cases and provide a preliminary list of requirements. In addition, based on the discussion of real-valued and complex data representations, we elaborate on potential coding technologies that range from approaches utilizing classical 2D coding technologies to holographic content-aware coding solutions. Finally, we address the problem of visual quality assessment of holographic data covering both visual quality metrics and subjective assessment methodologies.

Keywords

References

  1. ISO/IEC JTC1/SC29/WG1, JPEG PLENO Abstract and Executive Summary, WG1N6922, 68th JPEG Meeting, Sydney, AustraliaFebruary 2015, Feb 2015.
  2. T. Ebrahimi et al., JPEG Pleno: toward an efficient representation of visual reality, IEEE Multimedia 23 (2016), no. 4, 14-20. https://doi.org/10.1109/MMUL.2016.64
  3. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Part 1-ISO/IEC 21794-1 WD2.0, WG1N80009, 80th JPEG Meeting, Berlin, Germany, July 2018.
  4. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Call for Proposals on Light Field Coding, WG1N74014, 74th JPEG Meeting, Geneva, Switzerland, Jan. 2017.
  5. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Light Field Coding Verification Model 1.0, WG1N80028, 80th JPEG Meeting, Berlin, Germany, July 2018.
  6. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Part 2-ISO/IEC 21794-2 WD2.0, WG1N81057, 81th JPEG Meeting, Vancouver, Canda, Oct 2018.
  7. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Point Clouds-Use Cases and Requirements, WG1N80018, 80th JPEG Meeting, Berlin, Germany, July 2018.
  8. ISO/IEC JTC1/SC29/WG1, JPEG Pleno Holography-Overview of Holography 2.0, WG1N80042, 80th JPEG Meeting, Berlin, Germany, July 2018.
  9. A. Symeonidou et al., Three-dimensional rendering of computergenerated holograms acquired from point-clouds on light field displays, Applications of Digital Image Processing XXXIX, vol. 9971, SPIE, 2016, pp. 99710S.
  10. M. K. Kim Digital Holographic Microscopy: Principles, Techniques, and Applications, Springer Series in Optical Sciences, New York, USA, (2011).
  11. C. M. Vest, Holographic interferometry, John Wiley and Sons, New York, USA, 1979.
  12. D. M. Hoffman et al., Vergence-accommodation conflicts hinder visual performance and cause visual fatigue, J. Vis. 8 (2008), no. 3, 33-33.
  13. A. W. Lohmann et al., Space-bandwidth product of optical signals and systems, J. Opt. Soc. Am. A 13 (1996), no. 3, 470-473. https://doi.org/10.1364/JOSAA.13.000470
  14. J. W. Goodman, Introduction to Fourier optics, Roberts & Company, Greenwood Village, CO, USA, 2004.
  15. D. Blinder et al., Signal processing challenges for digital holographic video display systems, Signal Proces. Image Commun. 70 (2019), 114-130. https://doi.org/10.1016/j.image.2018.09.014
  16. M. Yamaguchi, N. Ohyama, and T. Honda, Holographic threedimensional printer: new method, Appl. Opt. 31 (1992), no. 2, 217-222. https://doi.org/10.1364/AO.31.000217
  17. ISO/IEC JTC1/SC29/WG1, JPEG systems website, 2018, Available at http://jpeg.org/jpegsystems (last accessed Sept. 2018).
  18. IEC TC 110, IEC TC 110 Website, 2018, Available at https://www.iec.ch/dyn/www/f?p=103:7:0::::FSP_ORG_ID,FSP_LANG_ ID:1313,25 (last accessed Nov. 2018).
  19. F. Temmermans et al., JPEG Privacy and Security framework for social networking and GLAM services, EURASIP J. Image Video Process. 2017 (2017), 68: 1-9.
  20. M. E. Lucente, Interactive computation of holograms using a look-up table, J. Electron. Imaging 2 (1993), no. 1, 28-34. https://doi.org/10.1117/12.133376
  21. R. W. Gerchberg and O. Saxton, A practical algorithm for the determination of the phase from image and diffraction plane pictures, Optik 35 (1971), no. 2, 237-246.
  22. I. Yamaguchi and T. Zhang, Phase-shifting digital holography, Opt. Lett. 22 (1997), no. 16, 1268-1270. https://doi.org/10.1364/OL.22.001268
  23. Y. Awatsuji et al., Parallel three-step phase-shifting digital holography, Appl. Opt. 45 (2006), no. 13, 2995-3002. https://doi.org/10.1364/AO.45.002995
  24. Z. Y. Alpaslan, H. S. El-Ghoroury, and P. Schelkens, Information processing challenges of full parallax light field displays, SPIE Photonics Europe 2018, Optics, Photonics and Digital Technologies for Multimedia Applications, vol. 10679, SPIE, 2018.
  25. T. J. Naughton et al., Compression of digital holograms for threedimensional object recognition, Algorithms and Systems for Optical Information Processing V, vol. 4471, International Society for Optics and Photonics, 2001, pp. 280-290.
  26. Y. Xing, B. Pesquet-Popescu, and F. Dufaux, Comparative study of scalar and vector quantization on different phase-shifting digital holographic data representations, 3DTV-Conf.: True Vision-Capture, Transmission Display 3D Video (3DTVCON), Budapest, Hungary, July 2-4, 2014, pp. 1-4.
  27. P. A. Cheremkhin and E. A. Kurbatova, Numerical comparison of scalar and vector methods of digital hologram compression, Proc. Holography, Diffractive Opt. Applicat. VII 10022 (2016), 1002227: 1-10.
  28. D. Blinder et al., Wavelet coding of off-axis holographic images, Applications of Digital Image Processing XXXVI, Proc. SPIE 8856, (2013), pp. 88561L: 1-2.
  29. J. P. Peixeiro et al., Holographic data coding: Benchmarking and extending HEVC with adapted transforms, IEEE Trans. Multimedia, 20 (2018), no. 2, 282-297. https://doi.org/10.1109/TMM.2017.2742701
  30. A. E. Shortt, T. J. Naughton, and B. Javidi, Compression of digital holograms of three-dimensional objects using wavelets, Opt. Express 14 (2006), no. 7, 2625-2630. https://doi.org/10.1364/OE.14.002625
  31. P. A. Cheremkhin and E. A. Kurbatova, Compression of digital holograms using 1-level wavelet transforms, thresholding and quantization of wavelet coefficients, Digital Holography and Three-Dimensional Imaging Jeju, Rep. of Korea, May 29 -June 1, 2017, pp. W2A.38: 1-3.
  32. L. T. Bang et al., Compression of digital hologram for threedimensional object using Wavelet-Bandelets transform, Opt. Express 19 (2011), no. 9, 8019-8031. https://doi.org/10.1364/OE.19.008019
  33. D. Blinder et al., JPEG 2000-based compression of fringe patterns for digital holographic microscopy, Opt. Eng. 53 (2014), no. 12, 123102: 1-13.
  34. Y. Xing et al., Adaptive nonseparable vector lifting scheme for digital holographic data compression, Appl. Opt. 54 (2015), no. 1, A98-A109. https://doi.org/10.1364/AO.54.000A98
  35. L. Onural, Diffraction from a wavelet point of view, Opt. Lett. 18 (1993), no. 11, 846-848. https://doi.org/10.1364/OL.18.000846
  36. M. Liebling, T. Blu, and M. Unser, Fresnelets: new multiresolution wavelet bases for digital holography, IEEE Trans. Image Process. 12 (2003), no. 1, 29-43. https://doi.org/10.1109/TIP.2002.806243
  37. E. Darakis and J. J. Soraghan, Use of Fresnelets for phase-shifting digital hologram compression, IEEE Trans. Image Process. 15 (2006), no. 12, 3804-3811. https://doi.org/10.1109/TIP.2006.884918
  38. M. V. Bernardo et al., Holographic representation: hologram plane vs. object plane, Signal Proces. Image Commun. 68 (2018), 193-206. https://doi.org/10.1016/j.image.2018.08.006
  39. M. Bernardo, A. Pinheiro, and M. Pereira, Benchmarking coding standards for digital holography represented on the object plane, Proc. SPIE 10679, Optics, Photonics, and Digital Technologies for Imaging Applications V, SPIE Photonics Europe, SPIE, Apr. 2018, pp. 6-8.
  40. D. H. Lee et al., Viewing angle dependent coding of digital holograms, Eur. Signal Process. Conf., Barcelona, Spain, Aug. 2011, pp. 1367-1371.
  41. K. Viswanathan, P. Gioia, and L. Morin, Wavelet compression of digital holograms: towards a view-dependent framework, Applications of Digital Image Processing XXXVI, vol. 8856, International Society for Optics and Photonics, 2013, pp. 88561N.
  42. A. El Rhammad et al., Color digital hologram compression based on matching pursuit, Appl. Opt. 57 (2018), no. 17, 4930-4942. https://doi.org/10.1364/AO.57.004930
  43. A. El Rhammad et al., View-dependent compression of digital hologram based on matching pursuit, Optics, Photonics, and Digital Technologies for Imaging Applications V, vol. 10679, International Society for Optics and Photonics, May 2018, pp. 106790L.
  44. D. Blinder et al., Unitary transforms using time-frequency warping for digital holograms of deep scenes, IEEE Trans. Comput. Imag. 4 (2018), no. 2, 206-218. https://doi.org/10.1109/TCI.2018.2813167
  45. D. Blinder et al., Open access database for experimental validations of holographic compression engines, Int. Quality Multimedia Experience (QoMEX), Pylos-Nestoras, Greece, May 26-29, 2015, pp. 1-6.
  46. A. Gilles et al., Hybrid approach for fast occlusion processing in computer-generated hologram calculation, Appl. Opt. 55 (2016), no. 20, 5459-5470. https://doi.org/10.1364/AO.55.005459
  47. A. Gilles et al., Computer generated hologram from multiviewplus-depth data considering specular reflections, IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Seattle, WA, USA, July 11-15, pp. 1-6.
  48. M. V. Bernardo et al., A digital hologram compression scheme for representation on the object plane, Proc. SPIE 10752, Applications of Digital Image Processing XLI, SPIE Optics + Photonics, SPIE, Aug. 2018.
  49. W. Lin and C.-C. Jay Kuo, Perceptual visual quality metrics: a survey, J. Vis. Commun. Image Represent. 22 (2011), no. 4, 297-312. https://doi.org/10.1016/j.jvcir.2011.01.005
  50. A. Ahar et al., A new similarity measure for complex valued data, Digital Holography Three-Dimensional Imaging, Jeju, Rep. of Korea, May 29-June 1 2017, pp. Tu1A.6: 1-3.
  51. A. Ahar, A. Barri, and P. Schelkens, From sparse coding significance to perceptual quality: a new approach for image quality assessment, IEEE Trans. Image Process. 27 (2018), no. 2, 879-893. https://doi.org/10.1109/TIP.2017.2771412
  52. A. Ahar et al., Performance evaluation of sparseness significance ranking measure (SSRM) on holographic content, Applicat. Lasers Sensing Free Space Commun, Orlando, FL, USA, 25-28. 2018, pp. Tu4A. 10: 1-2.
  53. ISO/IEC JTC1/SC29/WG1, Visual evaluation of JPEG 2000 color image compression performance, WG1N1583, 19th JPEG Meeting, Tokyo, Japan, 2001.
  54. F. De Simone et al., Subjective evaluation of JPEG XR image compression, Applications of Digital Image Processing XXXII, vol. 7443, International Society for Optics and Photonics, 2009, pp. 74430L.
  55. A. Artusi et al., Overview and evaluation of the JPEG XT HDR image compression standard, J. Real-Time Image Proc. (2015), 1-16.
  56. ISO/IEC 29170-2:2015 Information technology-Advanced image coding and evaluation-Part 2: Evaluation procedure for nearly lossless coding, Available at https://www.iso.org/standard/66094.html (last accessed June 16, 2018).
  57. A. Willeme et al., Overview of the JPEG XS core coding system subjective evaluations, SPIE Optics + Photonics 2018, Applications of Digital Image Processing XLI, vol. 10752, SPIE, San Diego, CA, USA, 2018.
  58. I. Viola and T. Ebrahimi, Quality assessment of compression solutions for ICIP 2017 Grand Challenge on light field image coding, Int. Conf. Multimedia Expo Workshops, San Diego, CA, USA, July 23-27, 2018.
  59. A. Symeonidou et al., Three-dimensional rendering of computer-generated holograms acquired from point-clouds on light field displays, SPIE Optics + Photonics, Applications of Digital Image Processing XXXIX, vol. 9971, SPIE, 2016, pp. 99710S.
  60. ISO/IEC JTC1/SC29/WG1, JPEG website, 2018, Available at http://jpeg.org (last accessed September 2018).

Cited by

  1. Reduction of Artefacts in JPEG-XR Compressed Images vol.19, pp.5, 2019, https://doi.org/10.3390/s19051214
  2. Recent Advances in the Processing and Rendering Algorithms for Computer-Generated Holography vol.8, pp.5, 2019, https://doi.org/10.3390/electronics8050556
  3. Digital holography data compression vol.11, pp.1, 2019, https://doi.org/10.5937/telfor1901052c
  4. 디지털 홀로그램 압축 기술 및 표준화 동향 vol.34, pp.6, 2019, https://doi.org/10.22648/etri.2019.j.340613
  5. Suitability analysis of holographic vs light field and 2D displays for subjective quality assessment of Fourier holograms vol.28, pp.24, 2019, https://doi.org/10.1364/oe.405984
  6. JPEG Pleno holography: scope and technology validation procedures vol.60, pp.3, 2019, https://doi.org/10.1364/ao.404305
  7. Recording a Hologram Transmitted over a Communication Channel on One Sideband vol.11, pp.23, 2019, https://doi.org/10.3390/app112311468