Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

High-resolution 3D Object Reconstruction using Multiple Cameras

다수의 카메라를 활용한 고해상도 3차원 객체 복원 시스템

  • 황성수 (한국과학기술원 전기 및 전자공학과) ;
  • 유지성 (한국과학기술원 전기 및 전자공학과) ;
  • 김희동 (한국과학기술원 전기 및 전자공학과) ;
  • 김수정 (한국과학기술원 전기 및 전자공학과) ;
  • 팽경현 (한국과학기술원 전기 및 전자공학과) ;
  • 김성대 (한국과학기술원 전기 및 전자공학과)
  • Received : 2013.06.13
  • Published : 2013.10.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a new system which produces high resolution 3D contents by capturing multiview images of an object using multiple cameras, and estimating geometric and texture information of the object from the captured images. Even though a variety of multiview image-based 3D reconstruction systems have been proposed, it was difficult to generate high resolution 3D contents because multiview image-based 3D reconstruction requires a large amount of memory and computation. In order to reduce computational complexity and memory size for 3D reconstruction, the proposed system predetermines the regions in input images where an object can exist to extract object boundaries fast. And for fast computation of a visual hull, the system represents silhouettes and 3D-2D projection/back-projection relations by chain codes and 1D homographies, respectively. The geometric data of the reconstructed object is compactly represented by a 3D segment-based data format which is called DoCube, and the 3D object is finally reconstructed after 3D mesh generation and texture mapping are performed. Experimental results show that the proposed system produces 3D object contents of $800{\times}800{\times}800$ resolution with a rate of 2.2 seconds per frame.

본 논문에서는 다수의 카메라들을 이용하여 3차원 공간상에 있는 물체에 대한 다중 시점 영상들을 획득하고, 그 영상들로부터 해당 3차원 물체에 대한 기하학적인 형상 및 질감 정보를 추정하여, 그 물체에 대한 고해상도 3차원 콘텐츠를 효율적으로 제작하는 새로운 시스템을 제안한다. 지금까지 다양한 다중 시점 영상 기반 3차원 객체 복원 시스템들이 제안되었지만 다중 시점 기반 3차원 객체 복원이 많은 메모리와 계산량을 필요로 하기 때문에 고해상도의 3차원 콘텐츠를 얻는 데에는 어려움이 있었다. 3차원 복원에 필요한 계산량 및 메모리량을 줄이기 위해 제안 시스템은 객체의 다중 시점을 촬영한 영상 내에서 객체가 존재할 수 있는 영역을 사전에 설정하여 객체 윤곽선 추출 과정을 빠르게 수행한다. 그리고 체인코드를 활용하여 실루엣 영상을 표현하고 3차원-2차원 투영 및 역투영 관계를 1차원 호모그래피를 통해 표현하여 객체의 비주얼 헐을 빠르게 계산한다. 복원된 3차원 객체의 기하정보는 3차원 선분 기반의 표현 기법인 DoCube를 활용하여 적은 데이터양으로 표현하였으며, 3차원 메시 생성 및 텍스쳐 맵핑을 수행하여 최종적인 3차원 객체를 생성한다. 실험 결과 제안 시스템이 $800{\times}800{\times}800$ 해상도의 3차원 객체 복원을 프레임 당 2.2초에 수행하는 것을 확인하였다.

Keywords

References

  1. A. Smolic, "3D Video and Free Viewpoint Video-From Capture to Display," Pattern Recognition, Vol. 44, pp.1958-1968, 2011. https://doi.org/10.1016/j.patcog.2010.09.005
  2. T. Kanade, and P. Rander, "Virtualized Reality: Constructing Virtual Worlds from Real Scenes," IEEE Multimedia, Vol. 4, no.1, pp.34-47, 1997.
  3. K.M. Cheung, and T. Kanade, "A Real Time System for Robust 3D Voxel Reconstruction of Human Motions," in Proc. Computer Vis. Pattern Recog., pp.714-720, USA, June 2000.
  4. S. Wurmlin, E. Lamboray, O.G. Staadt, M.H. Gross, "3D Video Recorder," in Proc. Pacific Conf. Computer Graph. Appl., pp.325-334, 2002.
  5. J. Carranza, C. Theobalt, M.A. Magnor, and H.-P. Seidel, "Free-Viewpoint Video of Human Actors," ACM Trans. Graphics, Vol.2, no.3, pp.569-577, July, 2003.
  6. T. Matsuyama, X. Wu, T. Takai, T. Wada, "Real-Time Dynamic 3-D Object Shape Reconstruction and High-Fidelity Texture Mapping for 3-D Video," IEEE Trans. Circuits and System for Video Technol., Vol.14, no.3, pp.357-369, March, 2004. https://doi.org/10.1109/TCSVT.2004.823396
  7. Hansung Kim, and Kwanghoon Sohn, "A 3D Modeling System Using Multiple Stereo Cameras," Journal of the Institute of Electronics Engineers of Korea, Vol.44, no.1, 제 44권 SP편 제 1호, 1-9쪽, January, 2007.
  8. J. Starck, and A. Hilton, "Surface Capture for Performance-Based Animation," IEEE Computer Graph. Appl., Vol. 27, no.3, pp.21-31, 2007.
  9. C. Theobalt, N. Ahmed, G. Ziegler, and H.-P. Seidel, "High-Quality Reconstruction from Multiview Video Streams," IEEE Signal Process. Magazine, Vol.24, no.6, pp.45-57, 2007. https://doi.org/10.1109/MSP.2007.4317463
  10. 윤상민, "다중 카메라를 이용한 3차원 움직임 복원 및 분석에 관한 연구," Koreal Multimedia Society, Vol. 13, no. 4, pp.18-23, December, 2009.
  11. E. Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seider, and S. Thrun, "Performance Capture from Sparse Multi-view Video," ACM Trans. Graphics, Vol.27, no.3, August, 2008.
  12. K. Li, Q. Dai, and W. Xu, "Markerless Shape and Motion Capture from Multiview Video Sequences," IEEE Trans. Circuits. Syst. Video. Technol., Vol. 21, no.3, pp.320-334, March, 2011. https://doi.org/10.1109/TCSVT.2011.2106251
  13. M. Straka, S. Hauswiesner, M. Ruther, and H. Bischof, "Rapid Skin:Estimating the 3D Human Pose and Shape in Real-Time," in Proc. 3D Imaging, Modeling, Processing, Visualization and Transmission, pp.41-48, 2012.
  14. A. Laurentini, "The Visual Hull Concept for Silhouette-based Image Understanding," IEEE Trans. PAMI, Vol. 16, no. 2, pp. 150-162, 1994. https://doi.org/10.1109/34.273735
  15. W. E. Lorensen, and H.E. Cline, "Marching cubes: A High Resolution 3D Surface Construction Algorithm," ACM SIGGRAPH Computer Graphics vol.21, no.4, pp.163-169, 1987. https://doi.org/10.1145/37402.37422
  16. H.D. Kim, S.S. Hwang, and S.D. Kim, "Geometric Properties of 3D Data Represented by DoSurface," in Proc. of International Conference on Electronics, Information and Communication, Jeongseon, Korea, February, 2012.
  17. S. Kim, H.D. Kim, W.J. Kim, and S.D. Kim, "Fast Computation of a Visual Hull," in Proc. ACCV, pp. 2075-2084, 2010.
  18. Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. Pattern Anal. Machine Intell., vol.22, no.11, pp.1330-1334, 2000. https://doi.org/10.1109/34.888718
  19. Kyunghyun Paeng, Sung Soo Hwang, Hee-Dong Kim, Sujung Kim, Jisung Yoo, and Seong Dae Kim, "Normalized Cross Correlation-based Multiview background Subtraction for 3D Object Reconstruction," Journal of the Institute of Electronics Engineers of Korea, Vol. 50, no.6, pp.228-237, June, 2013. https://doi.org/10.5573/ieek.2013.50.6.228
  20. F. van den Bergh, and V. Lalioti, "Software Chroma Keying in an Immersive Virtual Environment," South African Computer Journal, Vol. 24, pp.155-162, 1999.
  21. S.D. Kim, J.H. Lee, and J.K. Kim, "A New Chain-Coding Algorithm for Binary Images using Run-Length Codes," Computer Vision, Graphics, and Image Processing, vol.41, pp.114-128, 1988. https://doi.org/10.1016/0734-189X(88)90121-1

Cited by

  1. 3D Accuracy Analysis of Mobile Phone-based Stereo Images vol.19, pp.5, 2014, https://doi.org/10.5909/JBE.2014.19.5.677
  2. Bounding volume estimation algorithm for image-based 3D object reconstruction vol.3, pp.2, 2014, https://doi.org/10.5573/IEIESPC.2014.3.2.59