Knowledge-aware path: interpretable graph reasoning in proactive dialogue generation

doi:10.19682/j.cnki.1005-8885.2021.0005

The Journal of China Universities of Posts and Telecommunications ›› 2021, Vol. 28 ›› Issue (1): 1-9.doi: 10.19682/j.cnki.1005-8885.2021.0005

• Artificial intelligence • Next Articles

Knowledge-aware path: interpretable graph reasoning in proactive dialogue generation

School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2020-03-28 Revised:2020-08-09 Online:2021-02-28 Published:2021-03-28
Contact: Ya-Jing XU E-mail:xyj@bupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61702047)

Abstract

Abstract:

Proactive dialogue generates dialogue utterance based on a conversation goal and a given knowledge graph (KG). Existing methods combine knowledge of each turn of dialogue with knowledge triples by hidden variables, resulting in the interpretability of generation results is relatively poor. An interpretable knowledge-aware path (KAP) model was proposed for knowledge reasoning in proactive dialogue generation. KAP model can transform explicit and implicit knowledge of each turn of dialogue into corresponding dialogue state matrix, thus forming the KAP for dialogue history. Based on KAP, the next turn of dialogue state vector can be infered from both the topology and semantic of KG. This vector can indicate knowledge distribution of next sentence, so it enhances the accuracy and interpretability of dialogue generation. Experiments show that KAP model’s dialogue generation is closer to actual conversation than other state-of-the-art proactive dialogue models.

Key words: knowledge-aware path|graph reasoning|proactive dialogue system

Sun Yinan, Xu Yajing , Li Si, Guo Jun. Knowledge-aware path: interpretable graph reasoning in proactive dialogue generation[J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(1): 1-9.

References

1. Chen H S, Liu X R, Yin D W, et al. A survey on dialogue systems: Recent advances and new frontiers. ACM SIGKDD Explorations Newsletter, 2017, 19(2): 25-35.

2. Lian R Z, Xie M, Wang F, et al. Learning to select knowledge for response generation in dialog systems. Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19), 2019, Aug 10-16, Macao, China. Menlo Park ,CA, USA: American Association for Artificial Intelligence, 2019: 5081-5087.

3. Wu W Q, Guo Z, Zhou X Y, et al. Proactive human-machine conversation with explicit conversation goals. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL'19), 2019, Jul 28-Aug 2, Florence, Italy. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 3794-3804.

4. Zhou H, Chen C, Liu H, et al. Proactive knowledge-goals dialogue system based on pointer network. Natural Language Processing and Chinese Computing: Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC’19): Part 1, 2019, Oct 9-14, Dunhuang, China. LNCS 11838. Berlin, Germany: Springer, 2019: 724-735.

5. Vinyals O, Fortunato M, Jaitly N. Pointer networks. Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS'15): Vol 2, 2015, Dec 7-12, Montreal, Canada. Cambridge, MA, USA: MIT Press, 2015: 2692-2700.

6. Su H, Shen X Y, Zhang R Z, et al. Improving multi-turn dialogue modelling with utterance ReWriter. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Jul 28-Aug 2, Florence, Italy. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 22-31.

7. Shang L F, Lu Z D, Li H. Neural responding machine for short-text conversation. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP'15): Vol 1, 2015, Jul 16-21, Beijing, China. Stroudsburg, PA, USA: Association for Computational Linguistics, 2015: 1577-1586.

8. Long Y N, Wang J N, Xu Z, et al. A knowledge enhanced generative conversational service agent. Proceedings of the 6th Dialog System Technology Challenges (DSTC6) Workshop, 2017, Dec 10, Long Beach, CA, USA. 2017: 6p.

9. Ghazvininejad M, Brockett C, Chang M W, et al. A knowledge-grounded neural conversation model. Proceedings of the Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), the 30th Innovative Applications of Artificial Intelligence (IAAI’18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’18), 2018, Feb 2-7,New Orleans, LA, USA. Menlo Park ,CA, USA: American Association for Artificial Intelligence, 2018: 5110-5117.

10. Zhou H, Young T, Huang M L, et al. Commonsense knowledge aware conversation generation with graph attention. Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18), 2018, Jul 13-19, Stockholm, Sweden. Menlo Park ,CA, USA: American Association for Artificial Intelligence, 2018: 4623-4629.

11. Zhang Y J, Ren P J, de Rijke M. Improving background based conversation with context-aware knowledge pre-selection. arXiv preprint, arXiv:1906.06685, 2019.

12. Xu Z, Liu B Q, Wang B X, et al. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. arXiv preprint, arXiv:1605.05110, 2016.

13. Zhu W Y, Mo K X, Zhang Y, et al. Flexible end-to-end dialogue system for knowledge grounded conversation. arXiv preprint, arXiv:1709.04264, 2017.

14. Wang A, Liao M X, Lü P. A knowledge selection model in pointer-generator dialogue systems. Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery: Proceedings of the 15th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD’19): Vol 2, 2019, Jul, 20-22, Kunming, China. AISC 1075. Berlin, Germany: Springer, 2019: 451-458.

15. Gu J T, Lu Z D, Li H, et al. Incorporating copying mechanism in sequence-to-sequence learning. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL'16): Vol 1, 2016, Aug 7-12, Berlin, Germany. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 1631-1640.

16. Wang Y S, Liu C Y, Huang M L, et al. Learning to ask questions in open-domain conversational systems with typed decoders. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL'18): Vol 1, 2018, Jul 15-20, Melbourne, Australia. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 2193-2203.

17. Li R, Kahou S, Schulz H, et al. Towards deep conversational recommendations. Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NIPS’18), 2018, Dec 2-8, Montreal, Canada. 2018: 9725-9735.

18. Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18): Vol 1, 2018, Jun 1-6, New Orleans, LA, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 2227-2237.

19. Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14), 2014, Oct 25-29, Doha, Qatar. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 1532-1543.

20. Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT'19): Vol 1, 2019, Jun 2-7, Minneapolis, MN, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 4171-4186.

21. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Advances in Neural Information Processing Systems 30: Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS’17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY USA: Curran Associates Inc, 2017: 5998-6008.

22. Sukhbaatar S, Weston J, Fergus R. End-to-end memory networks. Advances in Neural Information Processing Systems 30: Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS’17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY USA: Curran Associates Inc, 2017: 6000–6010.

23. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS’1), 2011 Apr 11-13, Fort Lauderdale, FL, USA. 2011: 315-323.

24. Yao L L, Zhang Y Y, Feng Y S, et al. Towards implicit content-introducing for generative short-text conversation systems. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), 2017, Sept 7-11, Copenhagen, Denmark. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 2190-2199.

25. Papineni K, Roukos S, Ward T, et al. BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL’02), 2002, Jul 7-12, Philadelphia, PA, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2002: 311-318.

26. Paszke A, Gross S, Massa F, et al. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32: Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NIPS’19), 2019, Dec 8-14, Vancouver, Canada. 2019: 8024-8035.

27. Cui Y M, Che W X, Liu T, et al. Pre-training with whole word masking for Chinese BERT. arXiv preprint, arXiv:1906.08101, 2019.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Knowledge-aware path: interpretable graph reasoning in proactive dialogue generation

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended Articles

Metrics

Comments