Image Classification Algorithms Based on Neural Object Detectors and Structural Pattern Recognition

This study focuses on the use of object detection-based features in improving traditional methods of image classification. The technologies of image processing in the historical context were considered. The most effective methods of image processing, such as Alexnet, VGG, Resnet, Densenet, were selected. For the purpose of the study, Yelp Academic dataset, which contains multimodal information about businesses and restaurants represented on the platform, has been picked. Based on the data, two classification problems were formed: binary Business / Restaurant and multi-class multilabel Ambience classification (romantic, intimate, touristy, hipster, divey, upscale). The models were compared by the following metrics: ROC-AUC and F-measure (both tasks), Balanced Accuracy (Business / Restaurant only), Jaccard coefficient and Hamming Loss (Ambience only). The best results were achieved using Densenet-extracted vectors, with Support Vector Classifier for the Business / Restaurant task, and logistic regression for the Ambience task. Using R-CNN to extract vectors of maximum confidence and counts of objects for each type allows to increase accuracy by ~ 0.8% (Business / Restaurant), Jaccard coefficient by ~ 2.3% (Ambience) and F1-measure by ~ 0.85-1.5% (both tasks), compared to the traditional approaches.