The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Enhancing the Performance of Detecting Lip Feature Points for Facial Expression Recognition Based on AAM

AAM 기반 얼굴 표정 인식을 위한 입술 특징점 검출 성능 향상 연구

  • 한은정 (동국대학교 전자공학과) ;
  • 강병준 (한국전자통신연구원 휴먼인식기술연구팀) ;
  • 박강령 (동국대학교 전자공학과)
  • Published : 2009.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

AAM(Active Appearance Model) is an algorithm to extract face feature points with statistical models of shape and texture information based on PCA(Principal Component Analysis). This method is widely used for face recognition, face modeling and expression recognition. However, the detection performance of AAM algorithm is sensitive to initial value and the AAM method has the problem that detection error is increased when an input image is quite different from training data. Especially, the algorithm shows high accuracy in case of closed lips but the detection error is increased in case of opened lips and deformed lips according to the facial expression of user. To solve these problems, we propose the improved AAM algorithm using lip feature points which is extracted based on a new lip detection algorithm. In this paper, we select a searching region based on the face feature points which are detected by AAM algorithm. And lip corner points are extracted by using Canny edge detection and histogram projection method in the selected searching region. Then, lip region is accurately detected by combining color and edge information of lip in the searching region which is adjusted based on the position of the detected lip corners. Based on that, the accuracy and processing speed of lip detection are improved. Experimental results showed that the RMS(Root Mean Square) error of the proposed method was reduced as much as 4.21 pixels compared to that only using AAM algorithm.

AAM(Active Appearance Model)은 PCA(Principal Component Analysis)를 기반으로 객체의 형태(shape)와 질감(texture) 정보에 대한 통계적 모델을 통해 얼굴의 특징점을 검출하는 알고리즘으로 얼굴인식, 얼굴 모델링, 표정인식과 같은 응용에 널리 사용되고 있다. 하지만, AAM알고리즘은 초기 값에 민감하고 입력영상이 학습 데이터 영상과의 차이가 클 경우에는 검출 에러가 증가되는 문제가 있다. 특히, 입을 다문 입력얼굴 영상의 경우에는 비교적 높은 검출 정확도를 나타내지만, 사용자의 표정에 따라 입을 벌리거나 입의 모양이 변형된 얼굴 입력 영상의 경우에는 입술에 대한 검출 오류가 매우 증가되는 문제점이 있다. 이러한 문제점을 해결하기 위해 본 논문에서는 입술 특징점 검출을 통해 정확한 입술 영역을 검출한 후에 이 정보를 이용하여 AAM을 수행함으로써 얼굴 특징점 검출 정확성을 향상시키는 방법을 제안한다. 본 논문에서는 AAM으로 검출한 얼굴 특징점 정보를 기반으로 초기 입술 탐색 영역을 설정하고, 탐색 영역 내에서 Canny 경계 검출 및 히스토그램 프로젝션 방법을 이용하여 입술의 양 끝점을 추출한 후, 입술의 양 끝점을 기반으로 재설정된 탐색영역 내에서 입술의 칼라 정보와 에지 정보를 함께 결합함으로써 입술 검출의 정확도 및 처리속도를 향상시켰다. 실험결과, AAM 알고리즘을 단독으로 사용할 때보다, 제안한 방법을 사용하였을 경우 입술 특징점 검출 RMS(Root Mean Square) 에러가 4.21픽셀만큼 감소하였다.

Keywords

References

  1. S. J. Lee, S. B. Jung, J. W. Kwon, S. H. Hong, 'Face Detection and Recognition Using PCA,' in Proceedings of IEEE Region 10th Conference (TENCON'99) pp.84-87, 1999 https://doi.org/10.1109/TENCON.1999.818355
  2. Fangqi Tang, Benzai Deng, 'Facial Expression Recognition using AAM and Local Facial Features,' in Proceedings of International Conference on Natural Computation, pp.632-635, 2007 https://doi.org/10.1109/ICNC.2007.373
  3. Mehran Yazdi, Mehdi Seyf,i Amirhossein Rafati, Meghdad Asadi, 'Realtime Lip Contour Tracking For Audio-Visual Speech Recognition Applications,' in Proceedings of World Academy of Science Engineering and Technology, Vol.30, pp.164-167, 2008 https://doi.org/10.1007/3-540-61123-1
  4. 이종석, 심선희, 김소영, 박철훈, '제어되지 않은 조명 조건하에서 입술움직임의 강인한 특징추출을 이용한 바이모달 음성인식', Telecommunications Review, 14권, 1호, pp.123-134, 2004
  5. Ming-Hsuan Yang, D. J. Kriegman, and N. Ahuja, 'Detecting Faces in Images: A Survey," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.24, pp.34-58, 2002 https://doi.org/10.1109/34.982883
  6. Hadi Seyedarabi, Won Sook LEE, Ali Aghagolzadeh, 'Automatic Lip Tracking and Action Units Classification using Two-Step Active Contours and Probabilistic Neural Networks,' in Proceedings of Canadian Conference on Electrical and Computer Engineering, 2006 https://doi.org/10.1109/CCECE.2006.277379
  7. M. LiDevin, P. Delmas, P.Y. Coulon, F. Luthon, V. Fristot, 'Automatic Lip Tracking: Bayesian Segmentation and Active Contours in a Cooperative Scheme," in Proceedings of IEEE Conference on Multimedia, Computing and Systems, ICMCS'99, Fiorenza, Italy, June, 1999 https://doi.org/10.1109/MMCS.1999.779283
  8. Md. Mahbubur Rahman, Faisal Mahbub Chowdhury, Rajib Kumar Rana, Mohammad Fazleh Elahi, Md. Aminul Islam, 'A New Pseudo-automatic Outer Lip Contour Extraction Approach Based on RGB Components,' International Journal of the Computer, the Internet and Management, Vol.15, pp.94-101, 2007
  9. N. Eveno, A. Caplier, and P. Y. Coulon, 'A New Color Transformation for Lips Segmentation,' in Proceedings of IEEE Fourth Workshop On Multimedia Signal Processing, pp.3-8, 2001 https://doi.org/10.1109/MMSP.2001.962702
  10. R. M. Dansereau, C. Li and R. A. Goubran, 'Lip Feature Extraction Using Motion, Color and Edge Information,' in Proceedings of IEEE International Workshop on Haptic, Audio and Visual Environment and Applications, Sept., 2003 https://doi.org/10.1109/HAVE.2003.1244716
  11. Daehwan Kim, Jaemin Kim, Seongwon Cho, Yongsuk Jang, Sun-Tae Chung, Boo-Gyoun Kim, 'Progressive AAM Based Robust Face Alignment,' in Proceedings of World Academy of Science Engineering and Technology, Vol.21 pp.488-492, 2007
  12. OpenCV, http://opencvlibrary.sourceforge.net/ (accessed on 2009. 5. 27)
  13. M. B. Stegmann, B. K. Ersboll, R. Larsen, 'FAME - A Flexible Appearance Modelling Environment,' IEEE Transactions on Medical Imaging, Vol.22, Issue 10, pp.1319-1331, Oct. 2003 https://doi.org/10.1109/TMI.2003.817780
  14. T. Cootes, G. J. Edwards, and C. J. Taylor. 'Active Appearance Models,' in Proceedings of European Conference on Computer Vision, Vol.2, pp.484.498, 1998 https://doi.org/10.1109/34.927467
  15. Rafael C.Gonzalez, Richard E.Woods, 'Digital Image Processing', the 2nd edition, Prentice Hall, 2003
  16. T.F. Cootes, G.J. Edwards and C.J. Taylor, 'Active Appearance Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.681-685, June 2001 https://doi.org/10.1109/34.927467
  17. 한은정, 이의철, 고유진, 박강령, 민경하, 'AAM 모델에 기반한 다양한 입술 모양에 강인한 얼굴 특징 추출에 관한 연구', 대한전자공학회 2008년 추계학술대회, 31권, 2호, pp.643-644, 2008
  18. 이경희, 변혜란, '얼굴 요소의 영역 추출 및 snakes를 이용한 윤곽선 추출', 정보과학회논문지, 제27권, 제7호, 2000
  19. Logitec Camera, http://www.logitech.com/ (accessed on 2009. 5. 27)
  20. I. Matthews and S. Baker, 'Active Appearance Models Revisited,' International Journal of Computer Vision, Vol.60, No.2, pp.135-164, 2004 https://doi.org/10.1023/B:VISI.0000029666.37597.d3

Cited by

  1. A Feature Vector Generation Technique through Gradient Correction of an Outline in the Mouth Region vol.17, pp.10, 2014, https://doi.org/10.9717/kmms.2014.17.10.1141
  2. Real Time Lip Reading System Implementation in Embedded Environment vol.17B, pp.3, 2010, https://doi.org/10.3745/KIPSTB.2010.17B.3.227