Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Usability Evaluation of Artificial Intelligence Search Services Using the Naver App

인공지능 검색 서비스 활용에 따른 서비스 사용성 평가: 네이버 앱을 중심으로

  • 황신희 (연세대학교 디자인인텔리전스) ;
  • 주다영 (연세대학교 글로벌융합기술원)
  • Received : 2019.01.23
  • Accepted : 2019.06.15
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In the era of the 4th Industrial Revolution, artificial intelligence (AI) has become one of the core technologies in terms of the business strategy among information technology companies. Both international and domestic major portal companies are launching AI search services. These AI search services utilize voice, images, and other unstructured data to provide different experiences from existing text-based search services. An unfamiliar experience is a factor that can hinder the usability of the service. Therefore, the usability testing of the AI search services is necessary. This study examines the usability of the AI search service on the Naver App 8.9.3 beta version by comparing it with the search services of the current Naver App and targets 30 people in their 20s and 30s, who have experience using Naver apps. The usability of Smart Lens, Smart Voice, Smart Around, and AiRS, which are the Naver App beta versions of their artificial intelligence search service, is evaluated and statistically significant usability changes are revealed. Smart Lens, Smart Voice, and Smart Around exhibited positive changes, whereas AiRS exhibited negative changes in terms of usability. This study evaluates the change in usability according to the application of the artificial intelligence search services and investigates the correlation between the evaluation factors. The obtained data are expected to be useful for the usability evaluation of services that use AI.

4차 산업 혁명 시대에 인공지능은 IT 기업을 중심으로 기업들의 핵심 사업 전략이 되고 있다. 그리고 국내외 주요 포탈 기업들 또한, 인공지능 기반의 검색 서비스를 출시하고 있다. 인공지능 검색 서비스는 이미지 음성과 같은 비정형 데이터를 활용하며 검색 패러다임을 확장시켰다. 하지만 기존의 텍스트 기반의 검색 서비스와 다른 인터페이스를 제공한다. 익숙하지 않은 인터페이스는 서비스의 사용성을 저해할 수 있는 요소로, 인공지능 검색 서비스를 이용에 따른 사용성에 변화를 알아볼 필요가 있다. 본 연구는 네이버앱 8.9.3 베타버전을 사례로 인공지능 검색 서비스를 실험한다. 실험은 네이버앱 사용 경험이 있는 20대와 30대 30명을 대상으로, 네이버앱의 인공지능 검색 서비스인 스마트 렌즈, 스마트 보이스, 스마트 어라운드, AiRS 추천 콘텐츠의 사용성을 기존의 네이버앱 검색과 비교하여 평가한다. 실험분석 결과, 기존의 네이버앱 검색과 비교하여 통계적으로 유의미한 사용성 변화가 있는 것으로 나타났다. 스마트 렌즈, 스마트 보이스, 스마트 어라운드는 양(+)의 상관관계가, AiRS 추천 콘텐츠는 음(-)의 상관관계가 있었다. 본 연구는 인공지능 검색 서비스를 적용에 따른 사용성 변화를 평가하고 분석한 것으로, 추후 인공지능을 활용한 서비스의 사용성 평가 연구에 유용한 자료가 될 것으로 기대한다.

Keywords

References

  1. Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software Quality Journal, 11(4), 325-338. DOI: 10.1023/A:1025869312943
  2. Bok, K. S., Ahn, M. J., Im. J. T., & Yoo, J. S. (2014). Efficient location based social search considering time property in mobile environments. KIISE Transactions on Computing Practices, 20(4), 243-247.
  3. Cearley, D. W., Burke, B., Searle, S., & Walker. M. J. (2011). The top 10 strategic technology trends for 2018. Gartner. Retrieved from https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2018/
  4. Cho, Y. J., Han, S. H., Lee, K.T., Jung, G. S., & Cha, J. H. (2017). Study on the usability testing for voice-controlled user interface : Focused on IPTV usage. Proceedings of the Korean Information Science Society (pp. 1334-1336).
  5. Choi, Y. S. (2014). Comparative study on usability evaluation of text vs. graphic-based drugs retrieval system. Master's Thesis. University of Sungkyunkwan, Seoul.
  6. Choi, Y. R., & Kim, K. H. (2016). Artificial intelligence overview and application examples. Industrial Engineering Magazine, 23(2), 23-29.
  7. Elliot, B. & Andrews, W. (2017). A Framework for Applying AI in the Enterprise. USA: Gartner. ID: G00336031.
  8. Go, G. S., Jung, W. K., Shin, Y. G., Park, S. S., &Jang, D. S. (2011). A Study on Development of Patent Information Retrieval Using Textmining. Journal of the Korea Academia-Industrial Cooperation Society. 12(8), 3677-3688. https://doi.org/10.5762/KAIS.2011.12.8.3677
  9. Hong, J. S., Park, O. H., & Choi, C. H. (1996). Analysis of Survey Methods and Statistical Data. Seoul: Packyongsa.
  10. Internettrend. (2018). Search engine trends. Internettrend. Retrieved from http://www.internettrend.co.kr/
  11. Jang, P. S. (2016). [EU] 2016 Davos forum: What are our strategies for the forthcoming Fourth Industrial Revolution?. Science & Technology Policy, 26(2), 12-15.
  12. Kim, J. A. (2009). Usability evaluation of text-based search and visual search of a multidisciplinary library database. Journal of the Korean Society for Information Management, 26(3), 111-129. DOI: 10.3743/KOSIM.2009.26.3.111
  13. Makridakis, S. (2017). The forthcoming artificial intelligence (AI) revolution: Its impact on society and firms. Futures, 90, 46-60. DOI: 10.1016/j.futures.2017.03.006
  14. Mccarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955. AI Magazine, 27(4), 12. DOI: 10.1609/aimag.v27i4.1904
  15. Morville, P. (2004). User experience design. Semantic Studios. Retrieved from https://semanticstudios.com/user_experience_design/
  16. Nielsen, J. (1994). Usability inspection methods. In Conference companion on Human factors in computing systems (pp. 413-414). New York: ACM.
  17. Nielsen, J. (2000). Why you only need to test with 5 users. Nielsen Norman Group. Retrived from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  18. Russell, S. & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. New Jersey: Prentice-Hall.
  19. Shim, J. S. (2018). Naver has a strong AI passion. More than you think. Dongascience. Retrieved from http://dongascience.donga.com/news.php?idx=21978
  20. Sullivan, D. (2016). FAQ: All about the google rankbrain algorithm. Search Engine Land. Retrieved from http://searchengineland.com/faq-all-about-the-new-google-rankbrain-algorithm-234440
  21. Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1701-1708).
  22. Turing, A. M. (1995). Computing machinery and intelligence. Brian Physiology and Psychology, 213. DOI: 10.1007/978-1-4020-6710-5_3
  23. Unger, R., & Chandler, C. (2009). A project guide to UX design. India: Pearson.