Active Deep Learning

The paper shows the possibilities of using active learning methods on the example of images classification using convolutional neural networks. The issue is important since neural networks usually require a large sample size as well as the corresponding long learning time. Reducing the sample size will allow to significantly save on data labeling and possibly to reduce the learning time. Methods of active learning are described in the paper. Also, classification of the "20 Newsgroups"dataset articles using a multi-class logistic regression was demonstrated. Next, a series of experiments on the use of active learning was carried out, namely a pool￾based sampling strategy for a 2-layer convolutional neural network that classifies handwritten numbers from the MNIST dataset. Then, the tested methods were applied to a more complex task: images classification of the CIFAR10 dataset using a 3-layer convolutional network. The conducted experiments show that active learning methods work for the images classification problem with convolutional networks. Also the required sample size is reduced. The results of the experiments allow to conclude that the maximum entropy principle is the optimal approach for the described problem. The quality of sampling by the uncertainty level in the case of using the maximum entropy principle, within a reasonable sampling size, does not strongly depend on the sample size. It allows to reduce the learning time to the same one as in the whole dataset case. Attempts to improve the quality of the networks training through active learning were made. The results show that the principle of maximum entropy, used to select the batches at each step in the later stages of training, allows to obtain more stable and high quality predictions.