Simplification of Sentences Using Pre-trained Language Models

The task of simplifying the text includes several subtasks, where simplification at the sentence level was selected. In this formulation, the task is to get a simplified senetence from a complex sentence. The criteria for sentence complexity include the presence of complex grammatical structures, including participial and adverbial phrases, subordinate clauses, the presence of rare and ambiguous words, etc., in order to improve readability and understanding. However, for the Russian language, no large number of studies have been conducted on this problem. The aim of this work is to train various pre-trained language models on a dataset of paired sentences of the Russian language, and to achieve a good quality of simplification, comparable to modern works. As a result, the best method is the mT5 language model with control tokens, which achieved a SARI=39.375.