Journal of the Korea Society for Simulation (한국시뮬레이션학회논문지)

The Korea Society for Simulation (한국시뮬레이션학회)
권호 표지
DOI QR코드

DOI QR Code

The Battle Warship Simulation of Agent-based with Reinforcement and Evolutionary Learning

강화 및 진화 학습 기능을 갖는 에이전트 기반 함정 교전 시뮬레이션

  • 정찬호 (한국항공대학교 컴퓨터공학과) ;
  • 박철영 (한국항공대학교 컴퓨터공학과) ;
  • 지승도 (한국항공대학교 컴퓨터공학과) ;
  • 김재익 (국방과학연구소)
  • Received : 2012.10.05
  • Accepted : 2012.12.24
  • Published : 2012.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Due to the development of technology related to a weapon system and the info-communication, the battle system of a warship has to manage many kinds of human intervention tactics according to the complicated battlefield environment. Therefore, many kinds of studies about M&S(Modeling & Simulation) have been carried out recently. The previous M&S system based on an agent, however, has simply used non-flexible(or fixed) tactics. In this paper, we propose an agent modeling methodology which has reinforcement learning function for spontaneous(active) reaction and generation evolution learning Function using Genetic Algorithm for more proper reaction for warship battle. We experiment with virtual 1:1 warship combat simulation on the west sea so as to test validity of our proposed methodology. We consequently show the possibility of both reinforcement and evolution learning in a warship battle.

함정 전투체계는 무기체계, 정보통신 등의 기술 발전으로 인한 복잡한 전장 환경에 따라 인간이 개입하여 다양한 전술을 운용해야 한다. 따라서 에이전트 기반의 국방 M&S 시스템의 연구가 최근 들어 활발히 진행되고 있다. 그러나 현존하는 에이전트 기반 M&S 시스템은 고정된 전술을 적용하여 분석하는데 그치고 있다. 본 논문에서는 함정 교전에서 보다 적합한 대응을 찾기 위해 환경변화에 능동적으로 대처할 수 있도록 강화 학습 기능을 갖으며, 또한 유전 알고리즘을 이용하여 세대별 진화 학습 기능을 갖는 에이전트 모델링 방법론을 제안하였다. 타당성 검증을 위해 서해상에서 벌어지는 가상의 1:1 함정교전 시뮬레이션을 수행하였고, 이를 통해 함정 교전에 있어 강화 및 진화 학습이 가능함을 검증하였다.

Keywords

References

  1. A. Ilachinski, "Towards a Science of Experimental Complexity: An Artificial Life. Approach to Modeling Warfare", Proceedings of 5th Experimental Chaos Conference. Orlando. FL. 2000.
  2. Michael Babilot, "COMPARISON OF A DISTRIBUTED OPERATIONS FORCE TO A TRADITIONAL FORCE IN URBAN COMBAT", The Master's Thesis of Naval Postgraduate School, September, 2005.
  3. Easton A., and Barlow M., "CROCADILE: An Agent Based Distillation System Incorporating Aspects of Constructive Simulation", Proceedings of SimTect 2002, Melbourne Australia, 13-16 May, 2002.
  4. Yang A., H. A. Abbass and R. Sarker, "Evolving Agents for Network Centric Warfare", Proceedings of the 2005 GECCO Conference on Genetic and Evolutionary, pp. 25-29, June 2005.
  5. Lovgsdon J., D. Nash and M. Barnes, "OneSAF Tutorial", 2008 Defense Modeling and Simulation Conference (DMSC), Olando, FL, USA, March 2008.
  6. Yong-Jun You, Sung-Do Chi, Chan-Ho Jung et al, "A Study of Agent-based No-Human-in-the-Loop Battleship Warfare M&S System", The 7th Conference of marine weapon, pp. 29, 2008.
  7. Chan-Ho Jung, Yong-Jun You, Sung-Do Chi et al, "Manyto- Many Warship Combat Tactics Generation Methodology Using the Evolutionary Simulation", Journal of The Korea Society for Simulation, Vol. 19, No. 3, pp. 79-88, 2011. https://doi.org/10.9709/JKSS.2011.20.3.079
  8. M. Mitchell, and S. Forrest, "Genetic Algorithms and Artificial Life", ArtificialLife, MITPress, Cambrige,1995.
  9. Young-Kwang Kim, Jang-Se Lee, Sung-Do Chi, "Endomorphic Modeling of Intelligent Systems : Intelligent Card Game Players", Journal of KIISE, Vol. 26, No. 12, pp. 1507-1518, 1999.
  10. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning : a survey", Journal of Artificial Intelligence Research 4, 237-285, 1996.
  11. B. P. Zeigler, "Some Properties of Modified Demster- Shafer Operators in Rule-Based Inference Systems", Int. J. General Systems, 1987.
  12. Amandeep Singh Sidhu, Narendra S. Chaudhari, and Ghee Ming Goh, "Hierarchical Reinforcement Learning Model for Military Simulations", International Joint Conference on Neural Networks, Canada, July 2006.
  13. NGAI Chi Kit, "Reinforcement-Learning-Based Autonomous Vehicle Navigation in a Dynamically Changing Environment", for the Degree of Doctor of Philosophy at The University of Hong Kong, November 2007.
  14. Dong-Jin Lee, Hyo-Choong Bang, "Missile Evasive Strategies for Unmanned Aircrafts using Reinforcement Learning", The Conference of The Korean Society for Aeronautical & Space Sciences, pp 470-475, 2012.
  15. Dong-Jin Lee, Hyo-Choong Bang, "Adaptive Linear Quadratic Regulator for Unmanned Helicopters via Reinforcement Learning", The Conference of The Koxrean Society for Aeronautical & Space Sciences, pp 1343-1346, 2011.
  16. D.E. Goldberg, "Genetic Algorithm in Search, Optimization & Machine Learning", Addison-Wesley, 1989.
  17. A.S. Yilmaz, B.N. McQuay, H. Yu, A.S. Wu, and J.C. Sciortino. "Evolving Sensor Suites for Enemy Radar Detection", In Genetic and Evolutionary Computation Conference Proceedings, Part II, pp. 2384-2395. July 2003.
  18. S.W. Soliday. "A Genetic Algorithm model for mission planning and dynamic resource allocation of airborne sensors", In Proceedings, 1999 IRIS National Symposium on Sensor and Data Fusion, Laurel, MD, May 1999.
  19. Sung-Young Lee, Sung-Ho Jang, Jong-Sik Lee, "Modeling and Simulation of Optimal Path Considering Battlefieldsituation in the War-game Simulation", Journal of The Korea Society for Simulation, Vol. 19, No. 3, pp. 27-35, 2010.
  20. Jung-Hong Cho, Jung-Hae Kim, Jea-Soo Kim et al, "Optimal Acoustic Search Path Planning Based on Genetic Algorithm in Discrete Path System", Journal of The Korean Society of Ocean Engineers, Vol. 20, No. 1, pp. 69-76, 2006.