Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Image Based Human Action Recognition System to Support the Blind

시각장애인 보조를 위한 영상기반 휴먼 행동 인식 시스템

  • 고병철 (계명대학교 컴퓨터공학과) ;
  • 황민철 (계명대학교 컴퓨터공학과) ;
  • 남재열 (계명대학교 컴퓨터공학과)
  • Received : 2014.08.20
  • Accepted : 2014.11.07
  • Published : 2015.01.15
⟨ Previous Next ⟩

Abstract

In this paper we develop a novel human action recognition system based on communication between an ear-mounted Bluetooth camera and an action recognition server to aid scene recognition for the blind. First, if the blind capture an image of a specific location using the ear-mounted camera, the captured image is transmitted to the recognition server using a smartphone that is synchronized with the camera. The recognition server sequentially performs human detection, object detection and action recognition by analyzing human poses. The recognized action information is retransmitted to the smartphone and the user can hear the action information through the text-to-speech (TTS). Experimental results using the proposed system showed a 60.7% action recognition performance on the test data captured in indoor and outdoor environments.

본 논문에서는 시각장애인의 장면인식 보조를 위해, 귀걸이 형 블루투수 카메라와 행동인식 서버간의 통신을 통해 휴먼의 행동을 인식하는 시스템을 제안한다. 먼저 시각장애인이 귀걸이 형 블루투수 카메라를 이용하여 원하는 위치의 장면을 촬영하면, 촬영된 영상은 카메라와 연동된 스마트 폰을 통해 인식서버로 전송된다. 인식 서버에서는 영상 분석 알고리즘을 이용하여 휴먼 및 객체를 검출하고 휴먼의 포즈를 분석하여 휴먼 행동을 인식한다. 인식된 휴먼 행동 정보는 스마트 폰에 재 전송되고 사용자는 스마트 폰을 통해 text-to-speech (TTS)로 인식결과를 듣게 된다. 본 논문에서 제안한 시스템에서는 실내 외에서 촬영된 실험데이터에 대해서 60.7%의 휴먼 행동 인식 성능을 보여 주었다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. Statistics Korea (2014, June). Registered disabled persons of Korea [Online]. Available: http://www.index.go.kr/ (downloaded 2014, Aug.18)
  2. R. Kim, S. Park, S. Lee, H. Cho, and S. Ha, "Improvement of An Electronic Aid for the Blind using Ultrasonic and Acceleration Sensors," Journal of KIISE : Software and Applications, Vol. 36, No. 4, pp. 291-297, Apr. 2009. (in Korean)
  3. C. Kang, H. Cho, and B. Kim, "A Machine-tomachine based Intelligent Walking Assistance System for Visually Impaired Person," The Journal of Korea Information and Communications Society, Vol. 36, No. 3, pp. 287-296, Mar. 2011. (in Korean). https://doi.org/10.7840/KICS.2011.36B.3.287
  4. T-K. Kim, B-J. Choi, and B-S. Song, "A Machineto- machine based Intelligent Walking Assistance System for Visually Impaired person," The Korean Journal of Visual Impairment, Vol. 24. No. 2, pp. 43- 59, Feb. 2008.
  5. H-S Yoon (2013, March), Adaptive visual information guiding system for the blind [Online]. Available: http://www.etri.re.kr/etri/res/res_01020303.etri (downloaded 2014, Aug. 18)
  6. J. E. Son, J. H. Jung, B. C. Ko, and J. Y. Nam, "Part based human detection considering multi-view of human," Journal of KIISE :Computing Practices and Letters, Vol. 19, No. 11, pp. 596-600, Nov. 2013. (in Korean)
  7. J-H Jung, "Multi-View Object Detection with Sharing Local Features and Ensemble Learning of Random Forests," M.S thesis of Keimyung University, 2014.
  8. B.C. Ko, J-Y Kwak, and J-Y Nam, "Human Tracking in Thermal Images Using Adaptive Particle Filters with Online Random Forest Learning," Optical Engineering, Vol. 52, No. 11, pp. 1-14, Nov. 2013.
  9. J-H Hong, "Human action recognition in still images using action poselets and two-layer classification model," M.S thesis of Keimyung University, 2014.
  10. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 886-893, 2005.
  11. J. R. R. Uijlings, K. E. A. Sande, T. Gevers, and A. W. M. Smeulders, "Selective Search for Object Recognition," International Journal of Computer Vision, Vol. 104, No. 2, pp. 154-171, Sep. 2013. https://doi.org/10.1007/s11263-013-0620-5
  12. S. Maji, L. Bourdev, and J. Malik, "Action recognition from a distributed representation of pose and appearance," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3177- 3184, 2011.