中国邮电高校学报(英文) ›› 2020, Vol. 27 ›› Issue (2): 46-55.doi: 10.19682/j.cnki.1005-8885.2020.1006

• Artificial Intelligence • 上一篇    下一篇

Deep global-attention based convolutional network with dense connections for text classification

Tang Xianlun, Chen Yingjie, Xu Jin, Yu Xinxian   

  1. 1. School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
    2. School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
  • 收稿日期:2019-03-01 修回日期:2020-05-13 出版日期:2020-04-30 发布日期:2020-07-07
  • 通讯作者: Chen Yingjie, E-mail: potter1995@live.com E-mail:potter1995@live.com
  • 作者简介:Chen Yingjie, E-mail: potter1995@live.com
  • 基金资助:

    This work was supported by National Natural Science Foundation of China (61673079), Natural Science Foundation of Chongqing (cstc2018jcyjAX0160).

Deep global-attention based convolutional network with dense connections for text classification

Tang Xianlun, Chen Yingjie, Xu Jin, Yu Xinxian   

  1. 1. School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
    2. School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
  • Received:2019-03-01 Revised:2020-05-13 Online:2020-04-30 Published:2020-07-07
  • Contact: Chen Yingjie, E-mail: potter1995@live.com E-mail:potter1995@live.com
  • About author:Chen Yingjie, E-mail: potter1995@live.com
  • Supported by:
    This work was supported by National Natural Science Foundation of China (61673079), Natural Science Foundation of Chongqing (cstc2018jcyjAX0160).

摘要: Text classification is a classic task innatural language process (NLP). Convolutional neural networks (CNNs) have demonstrated its effectiveness in sentence and document modeling. However, most of existing CNN models are applied to the fixed-size convolution filters, thereby unable to adapt different local interdependency. To address this problem, a deep global-attention based convolutional network with dense connections (DGA-CCN) is proposed. In the framework, dense connections are applied to connect each convolution layer to each of the other layers which can accept information from all previous layers and get multiple sizes of local information. Then the local information extracted by the convolution layer is reweighted by deep global-attention to obtain a sequence representation with more valuable information of the whole sequence. A series of experiments are conducted on five text classification benchmarks, and the experimental results show that the proposed model improves upon the state of-the-art baselines on four of five datasets, which can show the effectiveness of our model for text classification.

关键词: deep learning, NLP, text classification, CNN, attention model

Abstract: Text classification is a classic task innatural language process (NLP). Convolutional neural networks (CNNs) have demonstrated its effectiveness in sentence and document modeling. However, most of existing CNN models are applied to the fixed-size convolution filters, thereby unable to adapt different local interdependency. To address this problem, a deep global-attention based convolutional network with dense connections (DGA-CCN) is proposed. In the framework, dense connections are applied to connect each convolution layer to each of the other layers which can accept information from all previous layers and get multiple sizes of local information. Then the local information extracted by the convolution layer is reweighted by deep global-attention to obtain a sequence representation with more valuable information of the whole sequence. A series of experiments are conducted on five text classification benchmarks, and the experimental results show that the proposed model improves upon the state of-the-art baselines on four of five datasets, which can show the effectiveness of our model for text classification.

Key words: deep learning, NLP, text classification, CNN, attention model