Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Modified Adaptive Random Testing through Iterative Partitioning

반복 분할 기반의 적응적 랜덤 테스팅 향상 기법

  • 이광규 ((주) NHN QA센터) ;
  • 신승훈 (아주대학교 정보통신전문대학원) ;
  • 박승규 (아주대학교 정보통신전문대학원)
  • Published : 2008.09.25
Download PDF
⟨ Previous Next ⟩

Abstract

An Adaptive Random Testing (ART) is one of test case generation algorithms that are designed to detect common failure patterns within input domain. The ART algorithm shows better performance than that of pure Random Testing (RT). Distance-bases ART (D-ART) and Restriction Random Testing (RRT) are well known examples of ART algorithms which are reported to have good performances. But significant drawbacks are observed as quadratic runtime and non-uniform distribution of test case. They are mainly caused by a huge amount of distance computations to generate test case which are distance based method. ART through Iterative Partitioning (IP-ART) significantly reduces the amount of computation of D-ART and RRT with iterative partitioning of input domain. However, non-uniform distribution of test case still exists, which play a role of obstacle to develop a scalable algerian. In this paper we propose a new ART method which mitigates the drawback of IP-ART while achieving improved fault-detection capability. Simulation results show that the proposed one has about 9 percent of improved F-measures with respect to other algorithms.

적응적 랜덤 테스팅 (Adaptive Random Testing, ART)은 입력 도메인 내의 오류 패턴을 순수 랜덤 테스팅 (Random Testing, RT)보다 좋은 효율로 찾아내기 위해 고안된 테스트 케이스 선택 알고리즘이다. 대표적인 ART 기법인 거리 기반 ART (Distance-based ART, D-ART)와 제한 영역 기반 ART (Restricted Random Testing, RRT) 둥은 좋은 성능을 보이기는 하지만, 테스트 케이스 선택에 필요한 많은 양의 거리 계산으로 인한 느린 테스트 케이스 생성과 거리 기반 방식의 사용으로 인한 테스트 케이스 분포의 불균일성이라는 단점을 가진다. 반복 분할 기반 ART (ART through Iterative Partitioning, IP-ART)는 입력 도메인을 반복 분할하는 방식을 통해 D-ART와 RRT가 가진 계산 부하를 크게 감소시켰다. 하지만 IP-ART의 경우에도 테스트 케이스 분포 문제는 여전히 존재하여 기법의 확장 적용에 대한 장애 요소로 작용하고 있다. 따라서 본 논문에서는 이와 같은 IP-ART의 단점 완화 및 성능 개선을 위한 방법을 제안하고, 실험을 통해 평균 9% 정도의 성능 향상을 확인하였다.

Keywords

References

  1. B. Hailpern, P. Santhanam, "Software Debugging, Testing, and Verification", IBM Systems Journal, vol. 41, no. 1, pp. 4-12, June 2002 https://doi.org/10.1147/sj.411.0004
  2. T. Y. Chen, D. H. Huang, Z. Q. Zhou, "Adaptive Random Testing Through Iterative Partitioning", in Proc. of the 11th Intl. Conf. on Reliable Software Tech., vol. 4006 of LNCS, pp. 155-166. Porto, Portugal, June 2006
  3. R. Hamlet, "Random testing", Encyclopedia of Software Engineering, Wiley, pp. 970-978, 1994
  4. P. S. Loo, W. K. Tsai, "Random Testing Revisited", Information and Software Technology, vol. 30, No. 9, pp. 402-417, September 1988 https://doi.org/10.1016/0950-5849(88)90037-7
  5. J. Mayer, C. Schneckenburger, "An Empirical Analysis and Comparison of Random Testing Techniques", in Proc. of the 2006 ACM/IEEE Intl. Symposium on Empirical Software Engineering (ISESE 2006), pp. 105-114, Rio de Janeiro, Brazil, September 2006
  6. J. Mayer, "Adaptive Random Testing with Randomly Translated Failure Region", in Proc. of the 1st Intl. Workshop on Random Testing (RT 2006), pp. 70-77, Maine, USA, July 2006
  7. T. Y. Chen, H. Leung, and I. K. Mak, "Adaptive Random Testing", in Proc. of the 9th Asian Computing Science Conf. (ASIAN 2004), vol. 3321 of LNCS, pp. 320-329. Chiang Mai, Thailand, December 2004
  8. K. P. Chan, T. Y. Chen, D. Towey, "Restricted Random Testing", in Proc. of the 7th European Conf. on Software Quality (ECSQ 2002), vol. 2349 of LNCS, pp. 321-330, Helsinki, Finland, June 2002
  9. T. Y. Chen, G. Eddy, R. Merkel, P. K. Wong, "Adaptive Random Testing Through Dynamic Partitioning", in Proc. of the 4th Intl. Conf. on Quality Software (QSIC 2004), pp. 79-86. Braunschweig, Germany, September 2004
  10. J. Mayer, C. Schneckenburger,, "Adaptive Random Testing with Enlarged Input Domain", in Proc. of the Sixth Intl. Conf. on Quality Software (QSIC 2006), pp. 251-258, Beijing, China , October 2006
  11. F. T. Chan, T. Y. Chen, I. K. Mak, and Y. T. Yu, "Proportional Sampling Strategy: Guidelines for Software Testing Practitioners", Information and Software Technology, vol. 38, no. 12, pp. 775-782, February 1996 https://doi.org/10.1016/0950-5849(96)01103-2
  12. T. Y. Chen, F. C. Kuo, R. G. Merkel, S. P. Ng, "Mirror Adaptive Random Testing", Information and Software Technology, vol. 46. issue 15, pp. 1001-1010, December 2004 https://doi.org/10.1016/j.infsof.2004.07.004
  13. J. Mayer, "Adaptive Random Testing by Bisection and Localization", in Proc. of the 5th Intl. Workshop on Formal Approaches to Testing of Software (FATES 2005), vol. 3997 of LNCS, pp. 72-86, Scotland, UK, July 2005
  14. T. Y. Chen, D. H. Huang, "Adaptive Random Testing by Localization", in Proc. of the 11th Asia-Pacific Software Engineering Conf. (APSEC 2004), vol. 00, pp. 292-298, Busan, Korea, December 2004
  15. J. Mayer, "Adaptive Random Testing by Bisection with Restriction", in Proc. of the 7th Intl. Conf. on Formal Engineering Methods (ICFEM 2005), vol. 3785 of LNCS, pages 251-263, Manchester, UK, November 2005
  16. J. Mayer, "Lattice-based Adaptive Random Testing", in Proc. of the 20th IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE 2005), pp. 333-336, CA, USA, November 2005