Machine Learning for News Summarization Task

In Master's dissertation corpora of russian news articles (RIA News agency) is investigated. The goal of the work is to try several summarization neural-models for news headlines generation and compare their quality. Comparable metric is rouge. In this work we use three models: encoder-decoder architecture with long short-term memory networks, transformer and recently publicated reformer. We briefly introduce structure of models listed above and key mechanisms of memory saving that are used in reformer: reversible connections and approximate attention calculation with locality-sensitive hashing (LSH). Models are trained in neural network frameworks: openNMT and Trax. Every model shows approximately the same quality. Generated news headlines look relatively friendly for human. Moreover in this work quality demonstrated by the original transformer are compared with quality of the transformer which computes attention only via LSH scheme.