ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Simple and effective neural coreference resolution for Korean language

  • Park, Cheoneum (AIRS Company, Hyundai Motor Group) ;
  • Lim, Joonho (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Ryu, Jihee (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Kim, Hyunki (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Lee, Changki (Computer Science, Kangwon National University)
  • Received : 2020.07.15
  • Accepted : 2021.01.22
  • Published : 2021.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

We propose an end-to-end neural coreference resolution for the Korean language that uses an attention mechanism to point to the same entity. Because Korean is a head-final language, we focused on a method that uses a pointer network based on the head. The key idea is to consider all nouns in the document as candidates based on the head-final characteristics of the Korean language and learn distributions over the referenced entity positions for each noun. Given the recent success of applications using bidirectional encoder representation from transformer (BERT) in natural language-processing tasks, we employed BERT in the proposed model to create word representations based on contextual information. The experimental results indicated that the proposed model achieved state-of-the-art performance in Korean language coreference resolution.

Keywords

Acknowledgement

This research was supported by the This work was supported by Institute for Information & Communications Technology Promotion (IITP) grants funded by the Korean government (MSIT) (2013-0-00131, Development of Knowledge Evolutionary WiseQA Platform Technology for Human Knowledge Augmented Services).

References

  1. H. Lee et al., Stanford's multi-pass sieve coreference resolution system at the conll-2011 shared task, in Proc. Conf. Comput. Natural Lang. Learn.: Shared Task (Portland, OR, USA), June 2011, pp. 28-34.
  2. V. Ng and C. Cardie, Improving machine learning approaches to coreference resolution, in Proc. Annu. Meet. Assoc. Comput. Linguistics (Philadelphia, PA, USA), July 2002, pp. 104-111.
  3. W. M. Soon, H. T. Ng, and D. C. Y. Lim, A machine learning approach to coreference resolution of noun phrases, Comput. Linguistics 27 (2001), no. 4, 521-544. https://doi.org/10.1162/089120101753342653
  4. N. Kwon, M. Polinsky, and R. Kluender, Subject preference in Korean, in Proc. West Coast Conf. Form. Linguistics (Somerville, MA, USA), Sept. 2006, pp. 1-14.
  5. D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint, CoRR, 2014, arXiv: 1409.0473.
  6. A. Vaswani et al., Attention is all you need, in Proc. Conf. Neural Inf. Process. Syst. (Long Beach, CA, USA), Dec. 2017, pp. 5998-6008.
  7. J. Devlin et al., Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint, CoRR, 2018, arXiv: 1810.04805.
  8. T. Mikolov et al., Recurrent neural network based language model, in Proc. Annu. Conf. Int. Speech Commun. Assoc. (Chiba, Japan), Sept. 2010, pp. 1045-1048.
  9. C. Park, K.-H. Choi, and C. Lee, Korean coreference resolution using the multi-pass sieve, J. KIISE 41 (2014), no. 11, 992-1005. https://doi.org/10.5626/JOK.2014.41.11.992
  10. C. Park et al., Korean coreference resolution with guided mention pair model using deep learning, ETRI J. 38 (2016), no. 6, 1207-1217. https://doi.org/10.4218/etrij.16.0115.0896
  11. K. Lee et al., End-to-end neural coreference resolution, CoRR, 2017, arXiv: 1707.07045.
  12. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9 (1997), no. 8, 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  13. K. Lee, L. He, and L. Zettlemoyer, Higher-order coreference resolution with coarse-to-fine inference, arXiv preprint, 2018, arXiv: 1804.05392.
  14. R. Zhang et al., Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering, arXiv preprint, CoRR, 2018, arXiv: 1805.04893.
  15. T. Lei, Y. Zhang, and Y. Artzi, Training RNNS as fast as CNNS, arXiv preprint, CoRR, 2017, arXiv: 1709.02755.
  16. R. K. Srivastava, K. Greff, and J. Schmidhuber, Highway networks, arXiv preprint, CoRR, 2015, arXiv: 1505.00387.
  17. R. Sennrich, B. Haddow, and A. Birch, Neural machine translation of rare words with subword units, arXiv preprint, CoRR, 2015, arXiv: 1508.07909.
  18. K. Clark and C. D. Manning, Improving coreference resolution by learning entity-level distributed representations, arXiv preprint, CoRR, 2016, arXiv: 1606.01323.
  19. M. Schuster and K. K. Paliwal, Bidirectional recurrent neural networks, IEEE Trans. Signal Process. 45 (1997), no. 11, 2673-2681. https://doi.org/10.1109/78.650093
  20. W. Wang et al., Gated self-matching networks for reading comprehension and question answering, in Proc. Annu. Meet. Assoc. Comput. Linguistics (Vancouver, Canada), July 2017, pp. 189-198.
  21. T. Dozat and C. D. Manning, Deep biaffine attention for neural dependency parsing, arXiv preprint, CoRR, 2016, arXiv: 1611.01734.
  22. D.-A. Clevert, T. Unterthiner, and S. Hochreiter, Fast and accurate deep network learning by exponential linear units (ELUs), arXiv preprint, CoRR, 2015, arXiv: 1511.07289.
  23. C. Park et al., Contextualized embedding- and character embedding-based pointer network for Korean coreference resolution, Annu. Conf. Hum. Lang. Technol. 2018 (2018), 239-242.
  24. M. Vilain et al., A model-theoretic coreference scoring scheme, in Proc. Conf. Message Underst. (Columbia, Maryland), Nov. 1995, pp. 45-52.
  25. A. Bagga and B. Baldwin, Algorithms for scoring coreference chains, in Proc. Int. Conf. Lang. Resour. Eval. (Granada, Spain), May 1998, pp. 563-566.
  26. X. Luo, On coreference resolution performance metrics, in Proc. Conf. Hum. Lang. Technol. Empirical Methods Nat. Lang. Process. (Vancouver, Canada), Oct. 2005, pp. 25-32.
  27. S. Pradhan et al., Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes, in Proc. Joint Conf. EMNLP and CoNLL: Shared Task (Jeju Island, South Korea), July 2012, pp. 1-40.
  28. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint, CoRR, 2014, arXiv: 1412.6980.