The Journal of the Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회논문지)

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-Modal User Distance Estimation System based on Mobile Device

모바일 디바이스 기반의 멀티 모달 사용자 거리 추정 시스템

  • 오병훈 (성균관대학교 전자전기컴퓨터공학과) ;
  • 홍광석 (성균관대학교 정보통신공학부)
  • Received : 2014.03.17
  • Accepted : 2014.04.11
  • Published : 2014.04.28
Download PDF
⟨ Previous Next ⟩

Abstract

This paper present the multi-modal user distance estimation system using mono camera and mono microphone basically equipped with a mobile device. In case of a distance estimation method using an image, we is estimated a distance of the user through the skin color region extraction step, a noise removal step, the face and eyes region detection step. On the other hand, in case of a distance estimation method using speech, we calculates the absolute difference between the value of the sample of speech input. The largest peak value of the calculated difference value is selected and samples before and after the peak are specified as the ROI(Region of Interest). The samples specified perform FFT(Fast Fourier Transform) and calculate the magnitude of the frequency domain. Magnitude obtained is compared with the distance model to calculate the likelihood. We is estimated user distance by adding with weights in the sorted value. The result of an experiment using the multi-modal method shows more improved measurement value than that of single modality.

본 논문에서는 모바일 디바이스에 기본적으로 제공되는 모노 카메라와 모노 마이크의 멀티 모달 입력을 통하여 사용자와 모바일 디바이스간의 거리를 추정하는 방법을 제안한다. 영상을 이용한 거리 추정은 모노 카메라로 입력되는 영상에서 피부색 영역을 추출하고, 노이즈를 제거한 후에 얼굴 영역 및 눈 영역을 검출하여 사용자의 거리를 추정한다. 음성을 이용한 거리 추정은 모노 마이크로 입력되는 음성으로부터 가장 큰 피크(Peak)를 선정하고, ROI(Region of Interest)를 지정한 후에 FFT(Fast Fourier Transform)을 수행하여 주파수 축에서의 크기(Magnitude)를 계산한다. 계산된 크기 값과 거리별 크기 값의 모델을 비교하여 거리 별 우도(Likelihood)를 계산하고, 정렬한 후 가중치를 주어 더함으로써 사용자의 거리를 추정한다. 실험결과 영상 및 음성을 멀티 모달 입력으로 이용하여 거리를 추정한 결과 단일 모달로 거리를 추정한 결과 보다 향상된 결과를 얻을 수 있었다.

Keywords

References

  1. A. Mulder, "Hand gestures for HCI", Technical Report 96-1, vol. Simon Fraster University, 1996.
  2. Ying Wu, Thomas S Huang, "Vision based, Gesture Recognition : A Review", Lecture Notes In Computer Science; Vol. 1739, Proceedings of the International Gesture Workshop Gesture-Based Communication in Human Computer Interaction, 1999.
  3. O. Bau, I. Poupyrev, A. Israr, C. Harrison, "TeslaTouch: Electrovibration for Touch Surfaces", UIST, 2010.
  4. Kitae Hwang, Jae-Moon Lee, "Preliminary Study on Soft Keyboard with Recommendation for Mobile Device", The Journal of The Institute of Internet, Broadcasting and Communication, Vol. 13, No. 6, pp. 137-145, December 2013. https://doi.org/10.7236/JIIBC.2013.13.6.137
  5. J. Y. Choi, S. J. Lee, B. C. Jeon, "Home Network Application using Android Mobile Platform", The Journal of The Institute of Internet, Broadcasting and Communication, Vol. 10, No. 4, pp. 35-40, August 2010.
  6. Y. S. Na, D. K. Chung, K. Y. Lee, "Smartphone Controller System using 3-D Acceleration Sensor", The Journal of The Institute of Internet, Broadcasting and Communication, Vol. 10, No. 4, pp. 23-28, August 2010.
  7. S. K, Oh, "Development of Intelligent Services and Analyzing User Behavior Information Using Smartphone", Journal of the Korea Academia -Industrial cooperation Society, Vol. 14, No1 2, pp. 6441-6446, 2013. https://doi.org/10.5762/KAIS.2013.14.12.6441
  8. Rahman, K.A and Hossain, M.S, "Person to Camera Distance Measurement Based on Eye-Distance", MUE'09, pp. 137-141, 2009.
  9. E. Georganti, T. May, S. van de Par, A Harma, and J. Mourjopoulos, "Speaker Distance Detection using a Single Microphone", IEEE Trans. Audio, Speech, and Language Processing, vol. 18, 2010.
  10. Paul Viola, Michael Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 511-518, 2001.
  11. Yoav Freund and Robert E. Schapire, "A decision theoretic generalization of on-line learning and an application to boosting", In Computational Learning Theory: Eurocolt 95, pp.23-37, 1997.
  12. P. Dhanalakshmi., S. Palanivel, V. Ramalingam, "Classification of audio signals using AANN and GMM," Applied soft computing, vol. 11, No. 1, pp.716-723, 2011. https://doi.org/10.1016/j.asoc.2009.12.033

Cited by

  1. Design of a Metamodel for the Development Process of a Mobile Application vol.15, pp.8, 2014, https://doi.org/10.5762/KAIS.2014.15.8.5248
  2. Development of Metrics to Measure Reusability of Mobile App. vol.15, pp.7, 2014, https://doi.org/10.5762/KAIS.2014.15.7.4500