KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Detection of Artificial Caption using Temporal and Spatial Information in Video

시·공간 정보를 이용한 동영상의 인공 캡션 검출

  • Received : 2012.05.31
  • Accepted : 2012.08.02
  • Published : 2012.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

The artificial captions appearing in videos include information that relates to the videos. In order to obtain the information carried by captions, many methods for caption extraction from videos have been studied. Most traditional methods of detecting caption region have used one frame. However video include not only spatial information but also temporal information. So we propose a method of detection caption region using temporal and spatial information. First, we make improved Text-Appearance-Map and detect continuous candidate regions through matching between candidate-regions. Second, we detect disappearing captions using disappearance test in candidate regions. In case of captions disappear, the caption regions are decided by a merging process which use temporal and spatial information. Final, we decide final caption regions through ANNs using edge direction histograms for verification. Our proposed method was experienced on many kinds of captions with a variety of sizes, shapes, positions and the experiment result was evaluated through Recall and Precision.

동영상에 포함되는 인공 캡션은 영상과 관계있는 의미정보를 포함한다. 이러한 영상을 표현하는 정보를 이용하기 위해 캡션을 추출하는 연구는 근래에 들어 활발히 진행되고 있다. 기존 방법들은 대부분 정지영상에서 캡션을 검출하였다. 하지만 동영상의 경우에는 유용한 시간정보가 있다. 따라서 본 연구는 이러한 시간정보를 사용한 캡션영역 검출방법을 제안한다. 먼저, 캡션후보영역 검출을 위해 문자출현맵을 생성하고, 후보영역 매칭 과정에서 지속후보영역을 검출한다. 검출된 지속후보영역의 소멸성 검사를 통해 캡션의 소멸 여부를 검출하고 소멸된 캡션 일 경우 시 공간정보에 의한 병합과정을 통해 캡션후보영역을 결정한다. 마지막으로 결정된 캡션후보영역을 검증하기 위하여 에지 방향 히스토그램을 이용한 신경망 인식기를 통하여 최종캡션영역을 검출한다. 실험을 위해 다양한 크기와 형태, 위치의 캡션을 포함하는 동영상에 대해 영역검출의 성능을 평가하고자 Recall과 Precision을 이용하여 제안하는 방법의 영역검출에 대한 효율성을 입증한다.

Keywords

References

  1. S.I.Joo, "Unstructured Caption Detection Using Temporal and Spatial Information in Video", Master degree Thesis, Soonsil Univ., 2010.
  2. R.Lienhart and F.Stuber, "Automatic Text Recognition in Digital Videos", In Proceedings of SPIE Image and Video Processing IV, Vol.2666, pp.180-188, September, 1996.
  3. R.Lienhart and W.Effelsberg, "Automatic Text Segmentation and Text Recognition for Video Indexing", Multimedia System, Vol.8, pp.69-81, January, 2000. https://doi.org/10.1007/s005300050006
  4. C.M.Lee and A.Kankanhalli, "Automatic extraction of characters in complex scene images", International Journal of Pattern Recognition Artificial Intelligence, Vol.9, No.1, pp. 67-82, 1995. https://doi.org/10.1142/S0218001495000043
  5. Y.Zhong, K.Karu and A.KJain, "Locating Text in Complex Color Images", Pattern Recognition, Vol.28, No.10, pp. 1523-1536, October, 1995. https://doi.org/10.1016/0031-3203(95)00030-4
  6. H.PLi and D.Doermann, "Automatic Text Detection and Tracking in Digital Video", IEEE Transactions on Image Processing, Vol.9, No.1, January, 2000.
  7. P.Shivakumara, T.Q.Phan and C.L.Tan, "New Wavelet and Color Features for Text Detection in Video," 20th International Conference on Pattern Recognition, pp.3996-3999, 2010.
  8. S.I.Hwang "A Study on Text Detection using DCT Coefficients in I-frame", Master degree Thesis, Yonsei Univ., 2002.
  9. J.Xi, X.H.Hua, X.R.Chen, L.Wenyin, H.J.Zhang, "A Video Text Detection and Recognition System", IEEE International Conference on Multimedia and Expo, pp.1080-1083, August 22-25, 2001.
  10. Q.X.Ye, Q.M.Huang, W.Gao and D.B.Zhao, "Fast and robust text detection in images and video frames", Image and Vision Computing, pp.565-576, Vol.23, No.6, 2005. https://doi.org/10.1016/j.imavis.2005.01.004
  11. P.Shivakumara, W.Huang and C.L.Tan, "Efficient video text detection using edge features", in Proc. ICPR, pp.1-4, 2008.
  12. C.H.Kwon, C.H.Shin, S.Y.Kim and S.H.Park, "Caption Detection Algorithm Using Temporal Information in Video", The Transactions of The Korean Institute of Electrical Engineers, Vol.53, No.8, pp.606-610, 2004.
  13. R.O.Duda, P.E.Hart and D.G.Stork, "Pattern Classfication", Wiley Interscience, Second Edition, Chapter 7.
  14. J.Davis and M.Goadrich, "The Relationship Between Precision Recall and ROC curves", Proceedings of the 23rd International Conference on Machine Learning(ICML 2006), pp.233-240, 2006.
  15. T.Fawcett, "ROC graphs : Notes and practical considerations for researchers." http://www.hpl.hp.com/personal/Tom_Fawcett/papers/index.html, 2003.