Diachronic Word Representation Shifts in Contextualized Embedding Models

In this thesis, we apply methods of automatic detection of temporal semantic shifts for Russian nouns and adjectives from pre-Soviet to Soviet times and from Soviet to post-Soviet times. These methods are based on contextualized word representation obtained with neural language models trained on diachronic corpora. Furthermore, we present manually annotated test sets for the task of tracing diachronic semantic shifts in Russian for two large-scale time period pairs. We tested two metrics of semantic shift that measure scores of semantic changes undergone by target words. Then we evaluated correlation between these scores and human judgments and qualitatively analyzed the results on specific examples. Results show that contextualized word representations can be used to detect diachronic semantic shifts related to cultural or intralinguistic factors such as metaphorization or nominalization.