Speech Markers of Intolerance and Computer Instruments of Their Detection

The paper is dedicated to the analysis of computer tools for automatic detection of lexical markers of speech intolerance. Two approaches are considered, the first one being the dictionary-oriented approach with manual editing capability and the second one being machine learning classification. The dictionary-based program compares words from lemmatized text with reference dictionaries and, when meeting an unfamiliar word, the program either ignores it or refers to the user for manual determination. The program also includes pre-processing and lemmatization of text files, allows the user to edit dictionaries, updates dictionaries during the usage and saves the detected markers in a separate text file. Machine learning classifiers, including Naive Bayes, Logistic Regression and Random Forest, were trained on a lemmatized corpus of short texts in Russian language, annotated by sentiment. The test texts were processed at the paragraph and sentence levels with the algorithm comparing the probability of the full text and the text with all occurrences of some word deleted being classified as a negative class. The words with the probabilities difference being the greatest are considered the markers of intolerance. The algorithms were tested on 10 texts of general topic from different sources with the results being evaluated on the bases of precision, recall, and F-measure metrics. All the algorithms experienced difficulties with the classification of invective lexics, occasionalisms, misspelled words, proper names, acronyms, and specific terms. Moreover, the classifiers also identified function words as markers. The dictionary approach has shown the best results due to the manual correction. Among the classifiers, Naive Bayes was the most effective, while the classification at the paragraph level showed greater precision and F-measure than the classification at the sentence level.