Slang Detection and Normalization

In this paper we'll suggest an approach on slang detection and normalization in Russian. The topic is most relevant to different resources on the Internet – social networks, forums, online shops. Normalization can be both useful to help moderate web-sites and for specialists in natural language processing while working with various sources of data. The proposed approach is to use Adaptive Skip-Gram neural network to remove the ambiguity of words that are supposed to have at least one non-normative sense. The main reason to choose model is the possibility to create more than one embedding for each word. For selecting a suitable synonym it was decided to use Word2Vec. The two models were trained on a large text corpus that contained different lexicon to make sure that we catch the slang meaning of polysemic words too. The list of slang vocabulary was collected from crowd-sourced resource for Russian teen slang. During the test, the results were evaluated and examples of good and bad substitutions were analyzed with presenting some ways to improve the quality.