ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Vision-Based Finger Action Recognition by Angle Detection and Contour Analysis

  • Received : 2010.06.08
  • Accepted : 2011.01.19
  • Published : 2011.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we present a novel vision-based method of recognizing finger actions for use in electronic appliance interfaces. Human skin is first detected by color and consecutive motion information. Then, fingertips are detected by a novel scale-invariant angle detection based on a variable k-cosine. Fingertip tracking is implemented by detected region-based tracking. By analyzing the contour of the tracked fingertip, fingertip parameters, such as position, thickness, and direction, are calculated. Finger actions, such as moving, clicking, and pointing, are recognized by analyzing these fingertip parameters. Experimental results show that the proposed angle detection can correctly detect fingertips, and that the recognized actions can be used for the interface with electronic appliances.

Keywords

References

  1. G. Hillebrand and K. Zuerl, "Finger Tracking for Virtual Agents," LNCS, vol. 4722, 2007, pp. 417-419.
  2. G. Hillebrand et al., "Inverse Kinematic Infrared Optical Finger Tracking," Int. Conf. Humans and Computers, 2006.
  3. P.G. Kry et al., "HandNavigator: Hands-On Interaction for Desktop Virtual Reality," ACM Symp. Virtual Reality Software Technol., Oct. 2008, pp. 53-60.
  4. R.Y. Wang and J. Popovic, "Real-Time Hand-Tracking with a Color Glove," Int. Conf. Computer Graphics Interactive Techn., 2009.
  5. S.H. Lee, J. Choi, and J. Park, "Interactive e-Learning System using Pattern Recognition and Augmented Reality," IEEE Trans. Consumer Electron., vol. 55, no. 2, May, 2009, pp. 883-890. https://doi.org/10.1109/TCE.2009.5174470
  6. F. Duca, J. Fredriksson, and M. Fjeld, "Real-Time 3D Hand Interaction: Single Webcam Low-Cost Approach," IEEE Workshop Trends Issues Tracking Virtual Environments, 2007, pp. 1-5.
  7. J. Segen and S. Kumar, "Look Ma, No Mouse!" Commun. ACM, vol. 43, no. 7, July 2000, pp. 102-109. https://doi.org/10.1145/341852.341869
  8. J. Segen and S. Kumar, "GestureVR: Vision-Based 3D Hand Interface for Spatial Interaction," Proc. 6th ACM Int. Conf. Multimedia, 1998, pp. 455-464.
  9. J.M. Kim and W.K. Lee, "Hand Shape Recognition Using Fingertips," 5th Int. Conf. Fuzzy Syst. Knowledge Discovery, 2008, pp. 44-48.
  10. D. Lee and Y. Park, "Vision-Based Remote Control System by Motion Detection and Open Finger Counting," IEEE Trans. Consumer Electron., vol. 55, no. 4, Nov. 2009, pp. 2308-2313. https://doi.org/10.1109/TCE.2009.5373803
  11. K. Oka, Y. Sato, and H. Koike, "Real-Time Fingertip Tracking and Gesture Recognition," IEEE Comp. Graph. Appl., vol. 22, no. 6, Dec. 2002, pp. 64-71. https://doi.org/10.1109/MCG.2002.1046630
  12. S.C. Crampton and M. Betke, "Counting Fingers in Real Time: A Webcam-Based Human-Computer Interface with Game Applications," Proc. Conf. Universal Access Human-Comput. Interaction, 2003, pp. 1357-1361.
  13. J. MacLean et al., "Fast Hand Gesture Recognition for Real-Time Teleconferencing Applications," Int. Workshop Recognition, Analysis Tracking of Faces Gestures Real Time Syst., 2001, pp. 133-140.
  14. R. Belaroussi and M. Milgram, "A Real Time Fingers Detection by Symmetry Transform Using a Two Cameras System," LNCS, vol. 5359, 2008, pp. 703-712.
  15. O. Gallo, S.M. Arteaga, and J.E. Davis, "A Camera-Based Pointing Interface for Mobile Devices," IEEE Int. Conf. Image Processing, 2008, pp. 1420-1423.
  16. B. Stenger et al., "Model-Based Hand Tracking Using a Hierarchical Bayesian Filter," IEEE Trans. Patt. Anal. Mach. Intell., vol. 28, no. 9, Sept. 2006, pp. 1372-1384. https://doi.org/10.1109/TPAMI.2006.189
  17. J. Alon et al., "A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation," IEEE Trans. Patt. Anal. Mach. Intell., vol. 31, no. 9, Sept. 2009, pp. 1685-1699. https://doi.org/10.1109/TPAMI.2008.203
  18. H. Ying et al., "Fingertip Detection and Tracking Using 2D and 3D Information," Proc. 7th World Congress Intell. Control Autom., 2008, pp. 1149-1152.
  19. C. Nolker and H. Ritter, "Detection of Fingertips in Human Hand Movement Sequences," Gesture Sign Language Human- Computer Interaction, 1997, pp. 209-218.
  20. A.M. Burns and B. Mazzarino. "Finger Tracking Methods Using EyesWeb," LNCS, vol. 3881, 2006, pp. 156-167.
  21. J. Hannuksela et al., "Motion-Based Finger Tracking for User Interface with Mobile Devices," IET 4th European Conf. Visual Media Production, 2007.
  22. A.A. Argyros and M.I.A. Lourakis, "Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse," LNCS, vol. 3979, 2006, pp. 40-51.
  23. J. Hannuksela et al., "Adaptive Motion-Based Gesture Recognition Interface for Mobile Phones," LNCS, vol. 5008, 2008, pp. 271-280.
  24. Y. Hirobe and T. Niikura, "Vision-Based Input Interface for Mobile Devices with High-Speed Fingertip Tracking," ACM Symp. User Interface, Software Technol., 2009, pp. 7-8.
  25. Z. Zivkovic, "Improved Adaptive Gaussian Mixture Model for Background Subtraction," Int. Conf. Pattern Recog., 2004, pp. 28-31.
  26. Y.J. Lee and D.H. Lee, "Research on Detecting Face and Hands for Motion-Based Game Using Web Camera," Int. Conf. Security Technol., 2008, pp. 7-12.
  27. A. Rosenfeld and E. Johnston, "Angle Detection on Digital Curves," IEEE Trans. Comp., vol. 22, no. 9, Sept. 1973, pp. 875- 878.
  28. A. Rosenfeld and J.S. Weszka, "An Improved Method of Angle Detection on Digital Curves," IEEE Trans. Comp., vol. 24, no. 9, Sept. 1975, pp. 940-941.
  29. R.E. Kalman, "A New Approach to Linear Filtering and Prediction Problems," J. Basic Eng., vol. 82-D, 1960, pp. 35-45.

Cited by

  1. Dual Autostereoscopic Display Platform for Multi-user Collaboration with Natural Interaction vol.34, pp.3, 2011, https://doi.org/10.4218/etrij.12.0211.0331
  2. CAMERA-BASED GESTURE TRACKING FOR 3D INTERACTION BEHIND MOBILE DEVICES vol.26, pp.8, 2011, https://doi.org/10.1142/s0218001412600087
  3. An Accurate Algorithm for the Identification of Fingertips Using an RGB-D Camera vol.3, pp.2, 2011, https://doi.org/10.1109/jetcas.2013.2256830
  4. A New Writing Experience: Finger Writing in the Air Using a Kinect Sensor vol.20, pp.4, 2013, https://doi.org/10.1109/mmul.2013.50
  5. Spatial Programming for Industrial Robots through Task Demonstration vol.10, pp.5, 2011, https://doi.org/10.5772/55640
  6. 깊이 영상 기반 손 영역 추적 및 손 끝점 검출 vol.18, pp.8, 2013, https://doi.org/10.9708/jksci.2013.18.8.065
  7. 산업용 지능형 로봇의 물체 인식 방법 vol.30, pp.9, 2013, https://doi.org/10.7736/kspe.2013.30.9.901
  8. 향상된 트래킹 시스템과 실시간 수성 사인펜을 위한 사실적 드로잉 vol.19, pp.2, 2011, https://doi.org/10.9708/jksci.2014.19.2.125
  9. Multimodal Interface Based on Novel HMI UI/UX for In-Vehicle Infotainment System vol.37, pp.4, 2011, https://doi.org/10.4218/etrij.15.0114.0076
  10. 3D Hand Gesture Analysis through a Real-Time Gesture Search Engine vol.12, pp.6, 2015, https://doi.org/10.5772/60045
  11. Vision based multi-user human computer interaction vol.75, pp.16, 2011, https://doi.org/10.1007/s11042-015-3061-z
  12. Near-Field Touch Interface Using Time-of-Flight Camera vol.28, pp.5, 2011, https://doi.org/10.20965/jrm.2016.p0759
  13. Skin Condition Estimation Using Mobile Handheld Camera vol.38, pp.4, 2011, https://doi.org/10.4218/etrij.16.0115.0942
  14. A Vision-Based Fingertip-Writing Character Recognition System vol.4, pp.4, 2011, https://doi.org/10.4236/jcc.2016.44014
  15. Body Segmentation using Gradient Background and Intra-Frame Collision Responses for Markerless Camera-Based Games vol.11, pp.1, 2016, https://doi.org/10.5370/jeet.2016.11.1.234
  16. Extracting hand articulations from monocular depth images using curvature scale space descriptors vol.17, pp.1, 2011, https://doi.org/10.1631/fitee.1500126
  17. Robust Fingertip Detection in a Complex Environment vol.18, pp.6, 2011, https://doi.org/10.1109/tmm.2016.2545401
  18. A New Ultrasonographic Image Displaying System to Support Vein Detection vol.29, pp.2, 2017, https://doi.org/10.20965/jrm.2017.p0306
  19. Vision-Based Fingertip Tracking Utilizing Curvature Points Clustering and Hash Model Representation vol.19, pp.8, 2017, https://doi.org/10.1109/tmm.2017.2691538
  20. Vision-Based Mid-Air Unistroke Character Input Using Polar Signatures vol.47, pp.6, 2017, https://doi.org/10.1109/thms.2017.2706695
  21. VEN-3DVE: vision based egocentric navigation for 3D virtual environments vol.13, pp.1, 2011, https://doi.org/10.1007/s12008-018-0481-9