Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

CNN-Based Hand Gesture Recognition for Wearable Applications

웨어러블 응용을 위한 CNN 기반 손 제스처 인식

  • Moon, Hyeon-Chul (Korea Aerospace University, School of Electronics and Information Engineering) ;
  • Yang, Anna (Korea Aerospace University, School of Electronics and Information Engineering) ;
  • Kim, Jae-Gon (Korea Aerospace University, School of Electronics and Information Engineering)
  • 문현철 (한국항공대학교 항공전자정보공학부) ;
  • 양안나 (한국항공대학교 항공전자정보공학부) ;
  • 김재곤 (한국항공대학교 항공전자정보공학부)
  • Received : 2018.01.12
  • Accepted : 2018.02.14
  • Published : 2018.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Hand gestures are attracting attention as a NUI (Natural User Interface) of wearable devices such as smart glasses. Recently, to support efficient media consumption in IoT (Internet of Things) and wearable environments, the standardization of IoMT (Internet of Media Things) is in the progress in MPEG. In IoMT, it is assumed that hand gesture detection and recognition are performed on a separate device, and thus provides an interoperable interface between these modules. Meanwhile, deep learning based hand gesture recognition techniques have been recently actively studied to improve the recognition performance. In this paper, we propose a method of hand gesture recognition based on CNN (Convolutional Neural Network) for various applications such as media consumption in wearable devices which is one of the use cases of IoMT. The proposed method detects hand contour from stereo images acquisitioned by smart glasses using depth information and color information, constructs data sets to learn CNN, and then recognizes gestures from input hand contour images. Experimental results show that the proposed method achieves the average 95% hand gesture recognition rate.

제스처는 스마트 글라스 등 웨어러블 기기의 NUI(Natural User Interface)로 주목받고 있다. 최근 MPEG에서는 IoT(Internet of Things) 및 웨어러블 환경에서의 효율적인 미디어 소비를 지원하기 위한 IoMT(Internet of Media Things) 표준화를 진행하고 있다. IoMT에서는 손 제스처 검출과 인식이 별도의 기기에서 수행되는 것을 가정하고 이들 모듈간의 인터페이스 규격을 제공하고 있다. 한편, 최근 인식률 개선을 위하여 딥러닝 기반의 손 제스처 인식 기법 또한 활발히 연구되고 있다. 본 논문에서는 IoMT의 유스 케이스(use case)의 하나인 웨어러블 기기에서의 미디어 소비 등 다양한 응용을 위하여 CNN(Convolutional Neural Network) 기반의 손 제스처 인식 기법을 제시한다. 제시된 기법은 스마트 글래스로 획득한 스테레오 비디오로부터 구한 깊이(depth) 정보와 색 정보를 이용하여 손 윤곽선을 검출하고, 검출된 손 윤곽선 영상을 데이터 셋으로 구성하여 CNN을 학습한 후, 이를 바탕으로 입력 손 윤곽선 영상의 제스처를 인식한다. 실험결과 제안기법은 95%의 손 제스처 인식율를 얻을 수 있음을 확인하였다.

Keywords

References

  1. A. Yang, S. Chun, H. Ko, J. G. Kim, "Hand gesture description for wearable applications in M-IoTW," ISO/IEC JTC1/SC29/WG11 M38526, Geneva, Swiss, May. 2016.
  2. A. Yang, S. Chun, and J-G. Kim, "Detection and recognition of hand gesture for wearable applications in IoMTW," In Proc. ICACT 2017, pp. 598-601, Feb. 2017.
  3. A. Yang, S. Chun, and J.-G. Kim, "Detection and Recognition of Hand Gesture for Wearable Applications in IoMTW," ICACT Trans. Advanced Communications Technology (TACT), vol. 6, no. 5, pp. 1046-1053, Sep. 2017.
  4. S. Mitra and T. Acharya, "Gesture recognition: A survey," IEEE Trans. Syst., Man, Cybern. C, vol. 37, no. 3, pp. 311-324, 2007. https://doi.org/10.1109/TSMCC.2007.893280
  5. S. Byun, S. Lee, G. Kim, S. Han, "Gesture recognition with wearable device based on deep learning," Broadcasting and Media Magazine, Vol.22, No.1, pp. 58-66, Jan.2017
  6. H. Moon, A. Yang, S. Chun, and J-G. Kim, "CNN-based hand gesture recognition for wearable applications," The Korean Institute of Broadcase and Media Engineers Conference, Seoul, Korea, pp. 58-59, 2017.
  7. Y. LeCun, K. Koray, and F. Clement, "Convolutional networks and applications in vision," In Proc. ISCAS 2010.
  8. M. Mitrea, "Working draft 2.0 of ISO/IEC 23093-1 IoMT Architecture," ISO/IEC JTC1/SC29/WG11 N17094, Torino, July 2017.
  9. A. Krizhevsky, I. Sutskever, and G. E. Hinton. "Imagent classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, 2012.
  10. I. Sutskever, J. Martens, G. Dahl, and G. Hinton, "On the importance of initialization and momentum in deep learning," In Proc, Int, Conf. Machine Learning (ICML), pp 1139-1147, Feb, 2013.
  11. Y. Lee, P. Moon, "A comparison and Analysis of deep learning frame-work," J. Korea Institute of Electron. Communi. Science (KIECS), vol 12, no.1, pp. 115-122, Feb, 2017.
  12. M. Han et al., "Visual hand gesture recognition with convolution neural network," In Proc. Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), 2016.