Methods of Automatic Noise Detection in News Feeds

Due to the development of the Internet, the amount of information generated by various media every day has increased significantly. This complicates the task of finding relevant information among the larger volume of all published news articles. This complicates the task of finding relevant information among the larger volume of all the news articles that have been published. In most user's news feeds the news feeds of users, a lot of “noisy” information comes in. For example, news about topics that do not interest a particular user or a large number of news about the same event. To get a complete picture of what is happening and do not reboot a person with a lot of information, news aggregators are created. In this paper, a new approach of clustering news feeds was described, which can be used to automate the processing of information collected from news feeds for its presentation in news aggregators. The task of text clustering for news aggregators is inherently reduced to two possible directions: 1. Thematic division of news - clustering of news, when one cluster should get news on the same topic. 2. Clustering news, when news of the same major event should be in the same cluster. In this work, a new approach of clustering the news feeds was described, which can help to simultaneously solve this kind of tasks. Here, we propose a method in which each individual news article is presented in the form of a semantic event graph that provides basic information about what happened. Further, these graphs are combined into one general and hierarchical method of defining communities on a graph, clusters nested into each other are distinguished. Clusters obtained at the level with the maximum value of modularity describe the division into topics, and clusters, with further separation who's value of the modularity, is about zero, describe individual events. This method was tested on data collected from various Russian media for the period of 02.2018-03.2018.