Journal of Korea Game Society (한국게임학회 논문지)

Korea Game Society (한국게임학회)
권호 표지
DOI QR코드

DOI QR Code

Research of Real-Time Emotion Recognition Interface Using Multiple Physiological Signals of EEG and ECG

뇌파 및 심전도 복합 생체신호를 이용한 실시간 감정인식 인터페이스 연구

  • Received : 2015.01.05
  • Accepted : 2015.03.02
  • Published : 2015.04.20
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a real time user interface that utilizes emotion recognition by physiological signals. To improve the problem that was low accuracy of emotion recognition through the traditional EEG(ElectroEncephaloGram), We developed a physiological signals-based emotion recognition system mixing relative power spectrum values of theta/alpha/beta/gamma EEG waves and autonomic nerve signal ratio of ECG (ElectroCardioGram). We propose both a data map and weight value modification algorithm to recognize six emotions of happy, fear, sad, joy, anger, and hatred. The datamap that stores the user-specific probability value is created and the algorithm updates the weighting to improve the accuracy of emotion recognition corresponding to each EEG channel. Also, as we compared the results of the EEG/ECG bio-singal complex data and single data consisting of EEG, the accuracy went up 23.77%. The proposed interface system with high accuracy will be utillized as a useful interface for controlling the game spaces and smart spaces.

뇌파 및 심전도 생체신호를 복합적으로 이용한 감정인식을 통한 실시간 사용자 인터페이스를 제안한다. 기존에 뇌파를 통한 감정인식의 문제점이었던 낮은 정확도를 개선하기 위해 뇌파의 Theta, Alpha, Beta, Gamma의 상대파워 값과 심전도의 자율신경계 비율을 혼합하는 복합 생체신호 감정 인식 시스템을 개발했다. 기쁨, 공포, 슬픔, 즐거움, 화남, 혐오에 해당하는 6가지 감정을 인식하기 위해 사용자별 확률 값을 저장하는 데이터 맵을 생성하고, 채널에 대응하는 감정 인식의 정확도를 향상시키기 위해 가중치를 갱신하는 알고리즘을 제안한다. 또한 뇌파로 구성된 단일 데이터와 뇌파/심전도 생체신호 복합 데이터의 실험 결과를 비교한 결과 23.77%의 정확도 증가를 보였다. 제안된 인터페이스 시스템은 높은 정확도를 통해 게임 및 스마트 공간의 제어에 필요한 인터페이스로 기기에 활용이 가능할 것이다.

Keywords

References

  1. KyungSik Kim, YoonJung Lee, DooNam Oh, "Usability Test of 'Paldokangsan3' a Walking Game for the Elderly", Journal of Korea Game Society, Vol. 15, No. 1, pp145-154, 2015. https://doi.org/10.7583/JKGS.2015.15.1.145
  2. Hae-Gul Pyun, Haeng-A An, Seongmin Yuk, Jinho Park, "A Gesture Interface based on Hologram and Haptics Environments for Interactive and Immersive Experiences", Journal of Korea Game Society, Vol. 15, No. 1, pp27-34, 2015. https://doi.org/10.7583/JKGS.2015.15.1.27
  3. Jonathan R. Wolpaw, Dennis J.. McFarlanda, Gregory W. Neatb, "An EEG-based brain-computer interface for cursor control", Electroencephalography and clinical neurophysiology, Vol. 78, No. 3, pp252-259, 1991. https://doi.org/10.1016/0013-4694(91)90040-B
  4. Jonathan R. Wolpaw, Dennis J. McFarland, "Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans", Proceedings of the National Academy of Sciences of the United States of America, Vol.101, No. 51, pp17849-17854, 2004. https://doi.org/10.1073/pnas.0403504101
  5. QiBin Zhao, LiQing Zhang, Andrzej Cichocki, "EEG-based asynchronous BCI control of a car in 3D virtual reality environments", Chinese Science Bulletin, Vol. 54, No. 1, pp78-87, 2009. https://doi.org/10.1007/s11434-008-0547-3
  6. Woestenburg, J. C., M. N. Verbaten, J. L. Slangen, "The removal of the eye-movement artifact from the EEG by regression analysis in the frequency domain.", Biological psychology, Vol. 16, No. 1 pp127-147, 1983. https://doi.org/10.1016/0301-0511(83)90059-5
  7. Shusterman, V., Barnea O., "Sympathetic nervous system activity in stress and biofeedback relaxation", Engineering in Medicine and Biology Magazine, IEEE Vol. 24, No. 2, pp52-57, 2005.
  8. Takayuki, H., Kiyoko Y., "The relaxation biofeedback system with computer and heart rate variability interaction", Technical Report of IEICE, pp35-38, 2003.
  9. Jennifer H., Rosalind P., "SmartCar: detecting driver stress", Proc. of the 15th IEEE Pattern Recognition 2000 International Conference, Vol. 4, pp218-221, 2000.
  10. Cai J., Liu G., Hao M., "The research on emotion recognition from ECG signal", Proc. of the IEEE ITCS 2009 International Conference, Vol. 1, pp. 497-500, 2009
  11. Harter, S., "A new self-report scale of intrinsic versus extrinsic orientation in the classroom: Motivational and informational components", Developmental psychology, Vol. 17, No. 3, pp300-312, 1981. https://doi.org/10.1037/0012-1649.17.3.300
  12. Stipek, D., "Motivation to learn: From theory to practice", Allyn and Bacon, 1993.
  13. Liu, Y., Sourina, O., Nguyen, M. K., "Real-time EEG-based human emotion recognition and visualization", Porc of the IEEE Cyberworlds 2010 International Conference, pp262-269, 2010.
  14. Khosrowabadi, R., Quek, H. C., Wahab, A., Ang, K. K., "EEG-based emotion recognition using self-organizing map for boundary detection" Proc. of the 20th IEEE ICPR 2010 International Conference, pp4242-4245, 2010.
  15. Petrantonakis, P. C., Hadjileontiadis, L. J., "EEG-based emotion recognition using hybrid filtering and higher order crossings" Proc. of the 3rd IEEE ACII 2009 International Conference, pp. 1-6, 2009.
  16. Ekman, P., "Facial expressions." Handbook of cognition and emotion pp226-232, 1999.
  17. Koelstra, S., Muhl, C., Soleymani, M., Lee, J. S., Yazdani, A., Ebrahimi, T., Patras, I., "Deap: A database for emotion analysis; using physiological signals" IEEE Trans. Affective Computing, Vol. 3, No. 1, pp18-31, 2012. https://doi.org/10.1109/T-AFFC.2011.15