Application of Deep Learning Models Using Structural Linguistic Information in NLP Tasks

This paper discusses ways of integration of structural linguistic information into modern neural network approaches to solve classification and generation NLP tasks. We propose a combination of GCN and RSTRecNN models to encode semantics, syntax, coreference relations and discourse simultaneously. Experiments are conducted for three popular classification benchmarks. The results of the base RSTRecNN model were improved in all cases. In addition, we present an ablation study to show the importance of using correct linguistic structures. Recent transformer-based approaches to NLG like GPT-2 can generate syntactically coherent original texts. However, these generated texts have a serious flaw: global discourse incoherence. This work presents an approach to estimate the quality of discourse structure. Empirical results confirm that the discourse structure of currently generated texts is inaccurate. We propose the research directions to correct it using discourse features during the fine-tuning procedure. Apart from that, we suggest a pipeline method consisting of two neural generation models to plan high-level discourse structure and present its preliminary results.