Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

An Emotion Recognition Method using Facial Expression and Speech Signal

얼굴표정과 음성을 이용한 감정인식

  • 고현주 (충북대학교 제어계측공학과) ;
  • 이대종 (충북대학교 컴퓨터정보통신 연구) ;
  • 전명근 (충북대학교 전기전자컴퓨터공학부)
  • Published : 2004.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we deal with an emotion recognition method using facial images and speech signal. Six basic human emotions including happiness, sadness, anger, surprise, fear and dislike are investigated. Emotion recognition using the facial expression is performed by using a multi-resolution analysis based on the discrete wavelet transform. And then, the feature vectors are extracted from the linear discriminant analysis method. On the other hand, the emotion recognition from speech signal method has a structure of performing the recognition algorithm independently for each wavelet subband and then the final recognition is obtained from a multi-decision making scheme.

본 논문에서는 사람의 얼굴표정과 음성 속에 담긴 6개의 기본감정(기쁨, 슬픔, 화남, 놀람, 혐오, 공포)에 대한 특징을 추출하고 인식하고자 한다. 이를 위해 얼굴표정을 이용한 감정인식에서는 이산 웨이블렛 기반 다해상도 분석을 이용하여 선형판별분석기법으로 특징을 추출하고 최소 거리 분류 방법을 이용하여 감정을 인식한다. 음성에서의 감정인식은 웨이블렛 필터뱅크를 이용하여 독립적인 감정을 확인한 후 다중의사 결정 기법에 외해 감정인식을 한다. 최종적으로 얼굴 표정에서의 감정인식과 음성에서의 감정인식을 융합하는 단계로 퍼지 소속함수를 이용하며, 각 감정에 대하여 소속도로 표현된 매칭 감은 얼굴에서의 감정과 음성에서의 감정별로 더하고 그중 가장 큰 값을 인식 대상의 감정으로 선정한다.

Keywords

References

  1. J. Lien, T. Kanade, C. Li, 'Detection, tracking, and classification of action units in facial expression,' Journal of Robotics and Autonomous Systems, Vol. 31, No.3, pp. 131-146, 2000 https://doi.org/10.1016/S0921-8890(99)00103-7
  2. M. Turk, A. Pentland, 'Eigenfaces for recognition,' Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-36, 1991 https://doi.org/10.1162/jocn.1991.3.1.71
  3. P. Penev, J. Atick, 'Local feature analysis: a general statistical theory for object representation,' Network : Computation in Neural Systems, Vol. 7, pp. 477-500, 1996 https://doi.org/10.1088/0954-898X/7/3/002
  4. P. Belhumeur, J Hespanha, D. Kriegman, 'Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,' IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No.7, pp. 711-720, 1997 https://doi.org/10.1109/34.598228
  5. C. Padgett, G. Cottrell, 'Representing face images for emotion classification,' Advances in Neural Information Processing Systems, Vol. 9, MIT Press. 1997
  6. Z. Zhang, M. Lyons, M. Schuster, S. Akamatsu, 'Comparison between geometry based and Gaborwavelets-based facial expression recognition using multi-layer perceptron,' Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 454-459, 1998 https://doi.org/10.1109/AFGR.1998.670990
  7. V.Kostov and S.Fukuda, 'Emotion in User Interface, Voice Interaction System,' IEEE Intl Conf. on Systems, Man, Cybernetics Representation, no. 2, pp. 798-803, 2000 https://doi.org/10.1109/ICSMC.2000.885947
  8. T. Moriyama and S. Oazwa, 'Emotion Recognition and Synthisis System on Speech,' IEEE Intl. Conference on Multimedea Computing and Systems, pages 840-844, 1999 https://doi.org/10.1109/MMCS.1999.779310
  9. L.C. Silva and P.C. Ng, Bimodal Emotion Recognition, Proceeding of the 4th International Conference on Automatic Face and Gesture Recognition, pp. 332-335, 2000 https://doi.org/10.1109/AFGR.2000.840655
  10. 김이곤, 배영철, '퍼지 로직을 이용한 감정인식 모델설계', 한국퍼지 및 지능시스템 춘계학술대회, 2000
  11. 심귀보, 박창현, '음성으로부터 감성인식 요소 분석' 퍼지 및 지능시스템학회 논문지, 2001
  12. P.Ekman and W.V. Friesen. 'Emotion in the human face System,' Cambridge University Press, San Francisco, CA, second edition, 1982
  13. 강현배, 김대경, 서진근, '웨이블릿 이론과 응용', 대우학술총서, 2001
  14. Hyung-Ji Lee, Wan-Su Lee, Jae-Ho Chung, 'Face recognition using Fisherface algorithm and elastic graph matching,' Image Processing, Volume: 1, 7-10 Oct. 2001 https://doi.org/10.1109/ICIP.2001.959216
  15. Fasel B, Luettin J, 'Recognition of asymmetric facial action unit activities and intensities,' Pattern Recognition, Proceedings. 15th International Conference on, Volume: 1, 3-7 Sept. 2000 https://doi.org/10.1109/ICPR.2000.905664
  16. Stephane Mallat, 'A wavelet tour of signal processing,' Academic press, 1999
  17. 이대종, 곽근창, 유정웅, 전명근, '웨이블렛 필터뱅크에 기반을 둔 강인한 화자식별 기법', 정보처리학회논문지C 제9-C권 제4호, 2002 https://doi.org/10.3745/KIPSTC.2002.9C.4.459
  18. Roger Jang, Chuen-Tsai Sun, 'Neuro-fuzzy and Soft computing,' Prentice-Hall International, 1997
  19. H. Kobayashi and F. Hara. 'Study on face robot for active human interface-mechanismsn of face robot an expression of 6 basic facial expression,' In IEEE Int'l Workshops on Robot and Human communication, pp.276-281, 1993 https://doi.org/10.1109/ROMAN.1993.367708
  20. H. Ushida, T. Takagi, and T. Yamaguchi. 'Recognition of facial expressions using conceptual fuzzy set,' In Proc. CVPR, pp594-599, 1993 https://doi.org/10.1109/FUZZY.1993.327421
  21. Katsuhiro Matsuno and saburo Tsuji, 'Recognizing human facial expressions in a potential field,' In Proc CVPR, pp 44-49, 1994 https://doi.org/10.1109/ICPR.1994.576873