Learnable Strategies for Uncertainty Estimation in Active Learning

It usually takes a lot of time for annotating training data for Natural Language Processing tasks, sequence tagging as an example, with acceptable quality. However, annotation budget can be drastically reduced with the help of recent advances in field of active learning and transformer-based deep learning architectures. In this paper we will investigate the performance of different variations of these architectures for named entity recognition task. We conduct an empirical study of various strategies for selections of examples, namely uncertainty and ensembles based on Monte Carlo dropout, with respect to these models to devise the best combinations for example annotation. We further study the recent efforts in imitation-based strategies for active learning and make attempts to find valuable features, learning algorithms and transfer solutions for scalable AL setups.