Detection and Analysis of Human Emotions through Speech

One of the tasks in the field of artificial intelligence is the task of recognizing the emotional state. Machine learning models achieved high results on simulated databases. However, at the moment there are practically no works devoted to the applied application of models that analyze emotions based on audio recordings. In this regard, the task was set: Investigation of methods for constructing feature spaces for speech audio data when solving applied problems based on the analysis of psycho-emotional coloring of speech and approbation of methods on an applied task of represent creditworthiness of clients using audio recordings of telephone conversations. The main problems of the task of assessing emotions based on audio recordings reviewed in the paper. A study was made of methods for constructing feature spaces, on the basis of which a set of features were selected for further use. Next, two models of emotion classification were built: SVM and CNN on two different databases. There are a conclusions about the good discriminating ability of the features obtained. Further, the developed technique was applied to the data of telephone conversations of bank customers. Based on the characteristics calculated from the audio recordings, the XGBoost classification model was built. According to the results of the work, it is possible to make a conclusion about the practical applicability of the analysis of the emotional state of bank customers. Signs obtained on the basis of audio recordings of customer conversations have a high discriminatory ability and can be used in the decision making process for issuing a loan. Correlation with existing customer data sources in the bank is minimal. This may indicate that the information obtained on the basis of speech represents an area of ​​knowledge about the client that is not used by the bank.