Developing of NLP Services for News Agency

This work is devoted to development of NLP services for news agency. The following problems are solved: multilabel classification of news, named entity recognition (including persons, organizations, locations, products), headline generation. Several neural network architectures are implemented for classification, allowing to achieve high classification quality ( 0.76 – 0.88 F1 score, depending on the level of categories). Named entity recognition is performed by both rule-based and neural-network based models. Two neural network architectures are implemented for headline generation: pointer generator, which allows copying out-of-vocab words from source to target, and RL summarizer, which can optimize not only language model likelihood, but also summarization metrics directly. Classification and NER models are successfully put in production and are used by RIA Novosti editors.