Deep neural networks in video-based face verification

This paper aims to develop a model underlying a video – based verification system. The verification process includes comparison of probe and gallery videos of a person to be verified and that one who the former states to be respectively. In order to compute similarity, each of videos is split into a set of frames, which are reduced to corresponding feature vectors and then aggregated into a single representation. The distance measured between representations of two aforementioned videos serves as a threshold for either acceptance or rejection decision – making. The accuracy of the result significantly depends on video representation, i.e. on data aggregation approach across video frames. The question of feature extraction is addressed by deep neural networks, which remain stable to varying both environment and face poses in a video and compresses it without leaving any necessary information. The paper carefully examines existing approaches to feature extraction in order to reveal the outperforming technique which would improve the future system that could be successfully introduced in number of organizations, thus partially automating the work of security departments without producing any costs for implementation as well as for maintenance. The paper is divided in three chapters as follows: the first includes a literature review on aforementioned topics, in the second chapter the implementation of real – time verification algorithm underlying verification system prototype OnlineFaceVerification is described. The last chapter provides results of practical experiments on YouTube Faces and Labeled Faces in the Wild datasets, which are intended to reveal the outperforming deep model among VGG and Lightened_CNN_C combined with several aggregation methods such as centroids and medoids algorithms as well as classifier fusion via ten – fold cross – validation.