Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Matching Points Filtering Applied Panorama Image Processing Using SURF and RANSAC Algorithm

SURF와 RANSAC 알고리즘을 이용한 대응점 필터링 적용 파노라마 이미지 처리

  • Kim, Jeongho (Department of Computer Science, Graduate School of Dankook University) ;
  • Kim, Daewon (Department of Multimedia Engineering, College of Engineering, Dankook University)
  • 김정호 (단국대학교 대학원 컴퓨터학과) ;
  • 김대원 (단국대학교 멀티미디어공학과)
  • Received : 2014.01.24
  • Accepted : 2014.03.31
  • Published : 2014.04.25
Download PDF
⟨ Previous Next ⟩

Abstract

Techniques for making a single panoramic image using multiple pictures are widely studied in many areas such as computer vision, computer graphics, etc. The panorama image can be applied to various fields like virtual reality, robot vision areas which require wide-angled shots as an useful way to overcome the limitations such as picture-angle, resolutions, and internal informations of an image taken from a single camera. It is so much meaningful in a point that a panoramic image usually provides better immersion feeling than a plain image. Although there are many ways to build a panoramic image, most of them are using the way of extracting feature points and matching points of each images for making a single panoramic image. In addition, those methods use the RANSAC(RANdom SAmple Consensus) algorithm with matching points and the Homography matrix to transform the image. The SURF(Speeded Up Robust Features) algorithm which is used in this paper to extract featuring points uses an image's black and white informations and local spatial informations. The SURF is widely being used since it is very much robust at detecting image's size, view-point changes, and additionally, faster than the SIFT(Scale Invariant Features Transform) algorithm. The SURF has a shortcoming of making an error which results in decreasing the RANSAC algorithm's performance speed when extracting image's feature points. As a result, this may increase the CPU usage occupation rate. The error of detecting matching points may role as a critical reason for disqualifying panoramic image's accuracy and lucidity. In this paper, in order to minimize errors of extracting matching points, we used $3{\times}3$ region's RGB pixel values around the matching points' coordinates to perform intermediate filtering process for removing wrong matching points. We have also presented analysis and evaluation results relating to enhanced working speed for producing a panorama image, CPU usage rate, extracted matching points' decreasing rate and accuracy.

다중의 영상을 이용하여 하나의 파노라마 영상을 제작하는 기법은 컴퓨터 비전, 컴퓨터 그래픽스 등과 같은 여러 분야에서 널리 연구되고 있다. 파노라마 영상은 하나의 카메라에서 얻을 수 있는 영상의 한계, 즉 예를 들어 화각, 화질, 정보량 등의 한계를 극복할 수 있는 좋은 방법으로서 가상현실, 로봇비전 등과 같이 광각의 영상이 요구되는 다양한 분야에서 응용될 수 있다. 파노라마 영상은 단일 영상과 비교하여 보다 큰 몰입감을 제공한다는 점에서 큰 의미를 갖는다. 현재 다양한 파노라마 영상 제작 기법들이 존재하지만, 대부분의 기법들이 공통적으로 파노라마 영상을 구성할 때 각 영상에 존재하는 특징점 및 대응점을 검출하는 방식을 사용하고 있다. 또한, 대응점을 이용한 RANSAC(RANdom SAmple Consensus) 알고리즘을 사용, Homography Matrix를 구하여 영상을 변환하는 방법을 사용한다. 본 논문에서 사용한 SURF(Speeded Up Robust Features) 알고리즘은 영상의 특징점을 검출할 때 영상의 흑백정보와 지역 공간 정보를 활용하는데, 영상의 크기 변화와 시점 검출에 강하며 SIFT(Scale Invariant Features Transform) 알고리즘에 비해 속도가 빠르다는 장점이 있어서 널리 사용되고 있다. SURF 알고리즘은 대응점 검출 시 잘못된 대응점을 검출하는 경우가 생긴다는 단점이 존재하는데 이는 RANSAC 알고리즘의 수행속도를 늦추며, 그로인해 CPU 사용 점유율을 높이기도 한다. 대응점 검출 오류는 파노라마 영상의 정확성 및 선명성을 떨어뜨리는 핵심 요인이 된다. 본 논문에서는 이러한 대응점 검출의 오류를 최소화하기 위하여 대응점 좌표 주변 $3{\times}3$ 영역의 RGB값을 사용하여 잘못된 대응점들을 제거하는 중간 필터링 과정을 수행하고, 문제해결을 시도하는 동시에 파노라마 이미지구성 처리 속도 및 CPU 사용 점유율 등의 성능 향상 결과와 추출된 대응점 감소율, 정확도 등과 관련한 분석 및 평가 결과를 제시하였다.

Keywords

References

  1. Daehyun Kim, Jongsoo Choi, "View Interpolation Algorithm for Continuously Changing Viewpoints in the Multi-panorama Based Navigation," IEIE Journal(SP), vol. 40, no. 6, pp. 141-148, Nov. 2003.
  2. Sehwan Kim, Kiyoung Kim, Woontack Woo, "Multiple Camera Calibration for Panoramic 3D Virtual Environment," IEIE Journal(CI), vol. 41, no. 2, pp. 137-148, Mar. 2004.
  3. Seung-Hyen Rhee, Jae Ho Lee, Sang-Gil Lee, and Moon Gi Kang, "Panoramic image reconstruction based on a camera motion model," KOSBE Journal, pp. 41-46, 1999.
  4. Jun-Seong Joe, Bum-jong Lee, and Jong-Seung Park, "Creating Full View Panorama Image from Multiple Images," KIISE Journal, vol. 34, no. 2(B), pp. 162-166, 2007.
  5. Zhihui Xiong, Maojun Zhang, Yunli Wang, Tun Li, and Kijun Li, "Fast Panorama Unrolling of Catadioptric Omni-Directional Images for Cooperative Robot Vision," Computer Supported Cooperative Work in Design, 11th International Conference on, DOI 10.1109/SCCWD.4281594, pp. 1100-1104, Apr. 2007.
  6. S. Kim and W. Woo, "Panoramic image-based virtual environment using the 3D generation," IPIU, vol. 13, pp. 111-116, 2003.
  7. Y. Seunbeom, J. HyeokJun and H. Chijeion, "Features-based panoramic image combinations," KICS Journal, vol. 35, no. 8, pp. 179-184, Sep. 2010.
  8. S. Richard, "Image Alignment and Stitching : A Tutorial," Journal of foundations and trends in computer graphics and vision, vol. 2, no. 1, Jan. 2006.
  9. J. M Morel and G. Yu, "ASIFT : A New Framework for Fully Affine Invariant Comparison," SIAM Journal on Imaging Sciences, vol. 2, no. 2, 2009.
  10. K. Mikolajczyk and C. Schmid, "Scale and Affine invariant interest point detectors," In IJC, vol. 60, no. 1, pp. 63-86, 2004.
  11. J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust wide baseline stereo from maximally stable extremal regions," In BMVC pp. 384-393, 2002.
  12. T. Tuytelaars and K. Mikolajczyk, "Local Invariant Feature Detectors," In CVG, vol. 3, no. 1, pp. 1-110, 2008.
  13. M. Brown and D. Lowe, "Automatic Panoramic Image Stitching using Invariant Features," International Journal of Computer Vision, vol. 74, no. 1, pp. 59-73, 2007 https://doi.org/10.1007/s11263-006-0002-3
  14. Chanil Park, Suhyun Lee, Yongjin Jeong, "A Hardware Design of Feature Detector for Realtime Processing of SIFT(Scale Invariant Feature Transform) Algorithm in Embedded Systems," IEIE Journal(SD), vol. 46, no. 3, pp. 86-95, Mar. 2009.
  15. H. Bay, T. Tuytelaars, and L. V. Gool, "Surf: Speeded up robust features," Computer Vision of Lecture Notes in Computer Science, Springer, vol. 3951. pp. 404-417, 2006.
  16. Sangwon Seo, Soowoong Jeong, Yunsang Han, Jongsoo Choi, Sangkeun Lee, "Efficient Homography Estimation for Panoramic Image Generation," vol. 50, no. 8, pp. 215-224, Aug. 2013. https://doi.org/10.5573/ieek.2013.50.8.215
  17. Luo Juan, Sungsik Shin, Hyun Ju Park, and Ou Bong Gwun, "Stitching for panorama based on SURF and Multi-band Blending," KMMS Journal, vol. 14, no. 2, pp. 201-209, 2011.
  18. Taehyub Kim, Yoonseok Choi, Bodam Nam, Hyunki Hong, "Tele-presence System using Homography-based Camera Tracking Method," IEIE Journal(CI), vol. 49, no. 3, pp. 27-33, May 2012.
  19. Jayhyun Im, Joonki Paik, "Feature-Based Panoramic Background Generation for Object Tracking in Dynamic Video," IEIE Journal(SP), vol. 45, no. 6, pp. 108-116, Nov. 2008.
  20. K. MinGoo, C. Wongook, and M. Seounbin, "SURF feature extraction algorithm for face recognition research," IEIE Journal(CI), vol. 48, no. 3, pp. 46-53, May 2011.
  21. M. Fischler and R. Bolles, "Random sample consunsus: A paradigm for model fitting applications to image analysis and automated cartography," Proc. Image Understanding Workshop, pp. 71-88, Apr. 1980.
  22. Son Jung-In, Kang Minsung, Sohn Kwanghoon, "View invariant image matching using SURF," KOSBE Conference, pp. 222-225, Aug. 2011.
  23. O. Chum, T. Pajdla, and P. Sturm, "The geometric error for homographies," Computer Vision and Image Understanding, vol. 97, no. 1, pp. 86-102, 2005. https://doi.org/10.1016/j.cviu.2004.03.004

Cited by

  1. Research of Panoramic Image Generation Using IoT Device with Camera for Cloud Computing Environment pp.1572-834X, 2018, https://doi.org/10.1007/s11277-018-5972-2
  2. Panoramic Image Stitching Using Feature Extracting and Matching on Embedded System vol.18, pp.5, 2014, https://doi.org/10.4313/teem.2017.18.5.273
  3. 재난재해 분야 드론 자료 활용을 위한 준 실시간 드론 영상 전처리 시스템 구축에 관한 연구 vol.18, pp.3, 2014, https://doi.org/10.7236/jiibc.2018.18.3.143