Transfer Learning for Information Extraction

Recently the Transfer Learning technique has become widely used in solving various problems in Natural Language Processing (NLP). The Cross-Lingual Transfer Learning (CLTL) approach, which involves Transfer Learning between texts in different languages, becomes extremely appropriate when working with small amount of labeled text data. Within this master's thesis there was investigated Cross-Lingual Transfer Learning approach on Named Entity Recognition (NER) and Relation Extraction (RE) tasks using various datasets in six languages. There were examined both simple (Word2Vec) and advanced (BERT, LaBSE, XLM-R) machine learning model architectures. The study shows the performance of the CLTL method in comparison with learning only on one language. As part of the work, there was implemented a Python framework that allows researchers to experiment with the Cross-Lingual Transfer Learning technique on NER and RE tasks. Keywords: machine learning, natural language processing, named entity recognition, relation extraction, cross-lingual transfer learning.