Adaptive Sampling for Boosting in Imbalanced Classification

Training classifiers with datasets which suffer of imbalanced class distributions is an important problem in data mining. This issue occurs when the number of examples representing the class of interest is much lower than the ones of the other classes. It presents in many real-world applications (there are more health people then ill and so on). The aim of this thesis is to modify boosting's algorithms to make it work well for imbalanced classification tasks. Imbalanced classification problem is considered in thesis. There are made experiments with different methods of data sampling on each iteration of deep boosting. In this methods the training instances are modified in such a way to produce a more or less balanced class distribution that allow classifiers to perform in a similar manner to standard classification. There is suggested new algorithm – Margin-Based Deep Smote Boosting, which combines advantages of Deep Boosting and SMOTE (sampling method, which is consisted in generating of synthetic objects of minority class), which works better then Smote Boosting и Deep Smote Boosting on imbalanced dataset (comparing is measured by F-score), uses significantly less trees in final ensemble (less then is used in Smote Boosting), works better then XGBOOST in imbalanced datasets (comparing is also measured by F-score). Suggested algorithm is based on Deep Boosting algorithm, which allows building trees of large depth without risk of overfitting, so there are much more complex separating surface. It assigns small weight to trees of large depth. It is possible by introducing regularization based on Rademacher complexity of a tree.