Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Fast and All-Purpose Area-Based Imagery Registration Using ConvNets

ConvNet을 활용한 영역기반 신속/범용 영상정합 기술

  • Received : 2016.06.10
  • Accepted : 2016.07.05
  • Published : 2016.09.15
⟨ Previous Next ⟩

Abstract

Together with machine-learning frameworks, area-based imagery registration techniques can be easily applied to diverse types of image pairs without predefined features and feature descriptors. However, feature detectors are often used to quickly identify candidate image patch pairs, limiting the applicability of these registration techniques. In this paper, we propose a ConvNet (Convolutional Network) "Dart" that provides not only the matching metric between patches, but also information about their distance, which are helpful in reducing the search space of the corresponding patch pairs. In addition, we propose a ConvNet "Fad" to identify the patches that are difficult for Dart to improve the accuracy of registration. These two networks were successfully implemented using Deep Learning with the help of a number of training instances generated from a few registered image pairs, and were successfully applied to solve a simple image registration problem, suggesting that this line of research is promising.

영역기반 영상정합은 미리 정의된 특징의 도움 없이 영상을 정합할 수 있기 때문에, 기계학습과 접목된다면 이론 상 다양한 영상정합 문제에 적용 가능하다. 그러나 신속한 정합을 위하여, 미리 정의된 특징을 탐지하여 패치 쌍 후보를 선정에 사용하는데, 이는 영역기반 방법의 적용성에 제약을 준다. 이를 해소하기 위하여 본 연구에서는 단순히 두 패치의 관련도 뿐만 아니라 두 패치가 어느 정도 공간 상 떨어져 있는지에 대한 정보를 제공하는 ConvNet Dart를 개발하였다. 이러한 정보를 기반으로 효율적으로 패치 쌍 탐색공간을 줄일 수 있었다. 추가로 Dart가 제대로 작동할 수 없는 영역을 식별하는 ConvNet Fad를 개발하여 정합의 정밀도를 높였다. 본 연구에서는 이들을 딥러닝으로 학습하였으며, 이를 위해 소수의 정합된 영상에서 다량의 예제를 생성하는 방법을 개발하였다. 마지막으로 단순한 영상정합 문제에 성공적으로 적용하여, 이러한 방법론이 작동하는 것을 보였다.

Keywords

Acknowledgement

Grant : SECRET PROJECT

Supported by : SECRET PROJECT

References

  1. S. Saxena and R. Singh, "A Survey of Recent and Classical Image Registration Methods," International Journal of Signal Processing, Image Processing and Pattern Recognition, Vol. 7, No. 4, pp. 167-176, 2014. https://doi.org/10.14257/ijsip.2014.7.4.16
  2. F. Oliveira and J. Tavares, "Medical image registration: a review," Computer Methods in Biomechanics and Biomedical Engineering, Vol. 17, No. 2, pp. 73-93, Mar. 2014. https://doi.org/10.1080/10255842.2012.670855
  3. Y. LeCun, Y. Bengion and G. Hinton, "Deep learning," Nature, Vol. 521, pp. 436-444, May. 2015. https://doi.org/10.1038/nature14539
  4. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, and L. Fei-Fei "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, Vol. 115, pp. 211-252, Apr. 2015. https://doi.org/10.1007/s11263-015-0816-y
  5. P. Fisher, A. Dosovitskiy, and T. Brox (2014, May 22) Descriptor matching with convolutional neural networks: a comparison to SIFT [Online]. Available: https://arxiv.org/abs/1405.5769 (downloaded 2014, May 22).
  6. J. Zbontar and Y. LeCun, "Computing the stereo matching cost with a convolutional neural network," Proc. of Computer Vision and Pattern Recognition 2015, pp. 1592-1599, 2015.
  7. S. Zagorukyo and N. Komodakis, "Learning to Compare Image Patches via Convolutional Neural Networks," Proc. of Computer Vision and Pattern Recognition 2015, pp. 4353-4361, 2015.
  8. G. Wu, M. Kim, Q. Wang, y. Gao, S. Liao and D. Shen, "Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images," Medical Image Computing, Computer-Asisted Intervention, Vol. 16, No. 2, pp. 649-656, Jun. 2013.
  9. D. Lowe, "Object recognition from local scale-invariant features," Proc. of the Internatl Conference on Computer Vision, pp. 150-1157, 1999.
  10. H. Vay, A. Ess, T. Tuytelaars and L. Van Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346-359, 2008. https://doi.org/10.1016/j.cviu.2007.09.014
  11. K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptor," IEEE Transactions on Pattern Analysis & Machine Intelligence, Vol. 27, No. 10, pp. 1615-1630, 2005. https://doi.org/10.1109/TPAMI.2005.188
  12. K, He, X. Zhang, S. Ren, and J. Sun Y., "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification," Proc. of the IEEE International Conference on Computer Vision, 2015.
  13. Y. Nesterov, "Gradient methods for minimizing composite functions," Mathematical Programming, Vol. 140, No. 1, pp. 125-161, 2013. https://doi.org/10.1007/s10107-012-0629-5
  14. M. Kumar, B. Packer, D. Koller, "Self-Paced Learning for Latent Variable Models," Advances in Neural Information Processing Systems, pp. 1189-1197, 2010.