Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Intelligent Spam-mail Filtering Based on Textual Information and Hyperlinks

텍스트정보와 하이퍼링크에 기반한 지능형 스팸 메일 필터링

  • 강신재 (대구대학교 컴퓨터ㆍIT공학부) ;
  • 김종완 (대구대학교 컴퓨터ㆍIT공학부)
  • Published : 2004.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper describes a two-phase intelligent method for filtering spam mail based on textual information and hyperlinks. Scince the body of spam mail has little text information, it provides insufficient hints to distinguish spam mails from legitimate mails. To resolve this problem, we follows hyperlinks contained in the email body, fetches contents of a remote webpage, and extracts hints (i.e., features) from original email body and fetched webpages. We divided hints into two kinds of information: definite information (sender`s information and definite spam keyword lists) and less definite textual information (words or phrases, and particular features of email). In filtering spam mails, definite information is used first, and then less definite textual information is applied. In our experiment, the method of fetching web pages achieved an improvement of F-measure by 9.4% over the method of using on original email header and body only.

본 논문은 텍스트 정보와 하이퍼링크에 기반한 2단계 지능형 스팸 메일 필터링에 관한 방법을 제시한다. 일반적으로 스팸 메일의 본문에는 텍스트 문장보다는 그림이 더 많이 포함되어 있기 때문에 단어의 블랙리스트와 같은 전형적인 방법으로 스팸 메일을 구분하기에는 많은 어려움이 따른다. 이러한 문제를 해결하기 위하여 본 논문에서는 스팸 메일에 포함되어 있는 하이퍼링크를 추출하여 해당 웹페이지를 가져온 후, 이를 확장된 형태의 메일 본문이라 간주하여 텍스트 정보를 추출하였다. 또한 스팸 메일을 구분하기 위한 정보를 두 가지로 구분하여 사용하였는데, 메일 송신자의 정보와 확실한 스팸 키워드 리스트를 확실한 정보군으로 구분하여 먼저 적용하고, 이보다 덜 명확한 정보들은 따로 구분하여 속성벡터를 만들어 SVM 알고리즘을 적용하였다. 실험결과 하이퍼링크를 통하여 웹페이지를 가져온 방법이 그냥 원본 메밀만 사용한 방법보다 F-measure 값이 평균 9.4% 의 성능향상을 보였다.

Keywords

References

  1. L. F. Cranor and B. A. LaMacchia, 'Spam!,' Communications of ACM, Vol. 41, Ko. 8, pp. 74-83, 1998 https://doi.org/10.1145/280324.280336
  2. M. Sahami, S. Dumais, D. Heckerman, and E. Horvitz, 'A bayesian approach to filtering junk e-mail,' In AAAI-98 Workshop on Learning for Text Categorization, pp. 55-62, 1998
  3. X. Carreras and L. Marquez, 'Boosting trees for anti-spam email filtering,', In Proceedings of ANLP-01, 4th International Conference on Recent Advances in Natural Language Processing, Tzigov Chark, BG, 2001
  4. J. Yang, V. Chalasani, and S. Park, 'Intelligent email categorization based on textual information and metadata,' IEICE Transactions on Information and System, Vol. E86-D, No .7, pp. 1280-1288, 2003
  5. H. Drucker, D. Wu, and V. Vapnik, 'Support Vector Machines for Spam Categorization,' IEEE Trans. on Neural Networks, Vol. 10, No. 5, pp. 1048-1054, 1999 https://doi.org/10.1109/72.788645
  6. T. Joachims, 'Text Categorization with Support Vector Machines: Learning with Many Relevant Features,' European Conference on Machine Learning (ECML), Claire Ndellec and Cline Rouveirol (ed), 1998
  7. I. Androutsopoulos, J. Koutsias, K. V. Chandrinos, G. Paliouras, and D. Spyropoulos, 'An evaluation of naive Bayesian anti-spam filtering,' In Proc. of the workshop on Machine Learning in theNew Information Age. 11th European Conference on Machine Learning. pp. 9-17, 2000
  8. I. Androutsopoulos, J. Koutsias, K. V. Chandrinos, and D. Spyropoulos, 'An experimental comparison of naive Bayesian and keyword-based anti-spam filtering with personal e-mail messages,'In Proc. of the 23 rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 160-167, Athens, Greece, 2000
  9. C. Apte, F. Damerau, and S. M. Weiss, 'Text Mining with Decision Trees and Decision Rules,' in Conference on Automated Learning and Discovery, Carnegie-Mellon University, June 1998
  10. A. McCallum and K. Nigam, 'A Comparison of Event Models for Naive Bayes Text Classification,' AAAI-98 Workshop on Learning for Text Categorization, 1998
  11. V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 1995
  12. O. de Vel, 'Mining E-mail Authorship,' Proc. Workshop on Text Mining, ACM International Conference on Knowledge Discovery and Data Mining (KDD'2000), Boston, 2000
  13. I. H. Witten and E. Frank, Data Mining: Practical machine learning tools and Techniques with java implementations, Morgan Kaufmann, 2000
  14. G. Salton, A. Wong, and C. S. Yang, 'A vector space model for automatic indexing,'Communication of the ACM, Vol. 18, No. 11, pp.613-620, 1975 https://doi.org/10.1145/361219.361220
  15. Y. Yang and J. P. Pedersen, 'A comparative study on feature selection in text categorization,' in Fourteenth International Conference onMachine Learning (ICML'97), pp. 412-420, 1997

Cited by

  1. A Big Data Learning for Patent Analysis vol.23, pp.5, 2013, https://doi.org/10.5391/JKIIS.2013.23.5.406
  2. Comparison Between Optimal Features of Korean and Chinese for Text Classification vol.25, pp.4, 2015, https://doi.org/10.5391/JKIIS.2015.25.4.386