KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time License Plate Detection Based on Faster R-CNN

Faster R-CNN 기반의 실시간 번호판 검출

  • 이동석 (전북대학교 IT 융합연구센터) ;
  • 윤숙 (목포대학교 멀티미디어공학과) ;
  • 이재환 (전북대학교 전자정보공학부) ;
  • 박동선 (전북대학교 전자공학부)
  • Received : 2016.10.04
  • Accepted : 2016.10.12
  • Published : 2016.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Automatic License Plate Detection (ALPD) is a key technology for a efficient traffic control. It is used to improve work efficiency in many applications such as toll payment systems and parking and traffic management. Until recently, the hand-crafted features made for image processing are used to detect license plates in most studies. It has the advantage in speed. but can degrade the detection rate with respect to various environmental changes. In this paper, we propose a way to utilize a Faster Region based Convolutional Neural Networks (Faster R-CNN) and a Conventional Convolutional Neural Networks (CNN), which improves the computational speed and is robust against changed environments. The module based on Faster R-CNN is used to detect license plate candidate regions from images and is followed by the module based on CNN to remove False Positives from the candidates. As a result, we achieved a detection rate of 99.94% from images captured under various environments. In addition, the average operating speed is 80ms/image. We implemented a fast and robust Real-Time License Plate Detection System.

자동차 번호판 검출 자동화(ALPD: Automatic License Plate Detection) 시스템은 효율적인 교통 관제를 위한 핵심 기술이며, 통행료 지불 시스템, 주차장 및 교통 관리와 같은 많은 응용에 사용되어 업무의 효율을 높이고 있다. 최근까지의 ALPD에 관한 연구에서는 주로 영상처리를 위해 설계된 기존의 특징들을 추출하여 번호판 검출에 사용해왔다. 이러한 종래의 방법은 속도에 이점은 있으나, 다양한 환경 변화에 따른 성능 저하를 보였다. 본 논문에서는 전반적인 성능을 향상시키기 위하여 Faster R-CNN과 CNN으로 구성되는 두 단 구조를 활용하는 방법을 제안한다. 이를 통해 동작 속도를 향상시키고, 다양한 환경변화에 강인하도록 구성하였다. 첫 번째 단계에서는 Faster R-CNN을 적용하여 번호판 영역 후보영역들을 선별하며, 두 번째 단에서 CNN을 활용하여 후보영역들 중에서 False Positives를 제거함으로써 검출률을 향상시켰다. 이를 통해 ZFNet을 기반으로 하여 99.94%의 검출률을 달성하였다. 또한 평균 운용시간은 80ms/image로써 빠르고 강인한 실시간 번호판 검출 시스템을 구현할 수 있었다.

Keywords

References

  1. P. Viola and M. Jones "Robust Real-time Face Detection," International Journal of Computer Vision, Vol.57, No.2, 2004.
  2. Md. M. K. Sarker, "License Plate Detection Using Cascade Adaboost and Heuristic Energy Map," Master's degree thesis, Chonbuk National University, 2013.
  3. N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.886-893, 2005.
  4. 최인수, 진문용, 박동선, "모폴로지 연산과 신명망 시스템을 이용한 차량 번호판 검출 및 숫자 인식," 스마트미디어학회, 2013 순천 정원엑스포 ICT 합동학술대회 논문집, 제2권, 제1호, pp.58-61, 2013.
  5. M. Sarfraz, M. J. Ahmed, and S. A. Ghazi, "Saudi Arabian license plate recognition system," in Proc. Int. Conf. Geom. Model. Graph., pp.36-41, 2003.
  6. A. M. Al-Ghaili, S. Mashohor, A. Ismail, and A. R. Ramli, "A new vertical edge detection algorithm and its application," in Proc. Int. Conf. Comput. Eng. Syst., pp.204-209, 2008.
  7. V. Kamat and S. Ganesan, "An efficient implementation of the Hough transform for detecting vehicle license plates using DSPs," in Proc. Real-Time Tech. Applicat. Symp., pp.58-59, 1995.
  8. P. Wu, H.-H. Chen, R.-J. Wu, and D.-F. Shen, "License plate extraction in low resolution video," in Proc. Int. Conf. Pattern Recognit., Vol.1, pp.824-827, 2006.
  9. K. Deb and J. Kang-Hyun, "HSI color based vehicle license plate detection," in Control, Automation and Systems, 2008. ICCAS 2008. International Conference on, pp.687-691, 2008.
  10. L. Dlagnekov, "License Plate Detection Using AdaBoost," San Diego, CA: Computer Science and Engineering Dept., 2004.
  11. S. Z. Wang and H. J. Lee, "A cascade framework for a real-time statistical plate recognition system," IEEE Trans. Inform. Forensics Security, Vol.2, No.2, pp.267-282, 2007. https://doi.org/10.1109/TIFS.2007.897251
  12. C. D. Nguyen, M. Ardabilian, and L. Chen, "Robust Car License Plate Localization Using a Novel Texture Descriptor," IEEE International Conference on Advanced Video and Signal Based Surveillance, pp.523-528, 2009.
  13. W. T. Ho, H. W. Lim, and Y. H. Tay, "Two-Stage License Plate Detection Using Gentle Adaboost and SIFT-SVM," First Asian Conference on Intelligent Information and Database Systems, pp.109-114, 2009.
  14. S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," arXiv preprint arXiv:1506.01497, 2015.
  15. M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional neural networks," in European Conference on Computer Vision (ECCV), 2014.
  16. Hui Li and Chunhua Shen, "Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs," arXiv arXiv:1601.05610v1 [cs.CV].
  17. G. E. Hinton, S. Osindero, and Y. W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., Vol.18, No. 7, pp.1527-1554, 2006. https://doi.org/10.1162/neco.2006.18.7.1527
  18. A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25, pp.1106-1114, 2012.
  19. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. "Going deeper with convolutions," in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR),, pp.1-9, 2015.
  20. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in ICLR, 2015.
  21. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Region based convolutional networks for accurate object detection and segmentation," TPAMI, 2015.
  22. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," in ECCV, 2014.
  23. R. Girshick, "Fast R-CNN," in IEEE International Conference on Computer Vision (ICCV), pp.1440-1448, 2015.
  24. J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W. Smeulders, "Selective search for object recognition," International Journal of Computer Vision (IJCV), pp.154-171, 2013.
  25. He, K., Zhang, X., Ren, S., and J. Sun, "Deep residual learning for image recognition," in CVPR, 2016.
  26. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting," JMLR, pp.1929-1958, 2014.
  27. Moon Yong Jin, Jong Bin Park, Dongsuk Lee, Dong Sun Park, "Real-Time Vehicle License Plate Recognition System Using Adaptive Heuristic Segmentation Algorithm," in KIPS Tr. Software and Data Eng., Vol.3, No.9, pp.361-368, 2014. https://doi.org/10.3745/KTSDE.2014.3.9.361