Deep Learning Technologies in Group Emotion Monitoring for Video Analytics System

In this paper the group-level emotion classification problem in video analytic systems of supermarkets is addressed. The requirements for video analytic system for the marketing needs are gathered. New algorithm for group-level emotion recognition is proposed. MTCNN face detector is applied to obtain facial regions on each video frame. Next, off-the-shelf image features are extracted from each located face using preliminary trained convolutional neural networks. The features of the whole frame are computed as a mean average of image embeddings of individual faces. The resulted frame features are recognized with an ensemble of state-of-the-art classifiers computed as a weighted sum of their outputs. Experimental results with EmotiW 2017 dataset demonstrate that the proposed approach achieves 75.5% accuracy which is 8-23% more accurate when compared to the conventional group-level emotion classifiers. Program module is implemented according to the requirements.