Applying Adaptive Learning Rate for Learning to Rank Problem

This research work considers very important machine learning problem of automatic ordering of search engine results with respect to a relevance score for a given search query. This problem is essential for a lot of different applications including web-search, automatic recommendation systems etc. Nowadays one of the most successful approaches to learning to rank is the use of additive ensembles of decision trees. Particularly the LambdaMART algorithm learns decision trees ensembles by optimizing pairwise loss function which gradient is multiplied by absolute difference in discrete metrics value. Empirical study has shown that ranking function reaches overfitting at different iterations of the algorithm for different queries because of diversity of information contained in them. This result leads to an assumption that adaptation of learning rate of gradient boosting with respect to every query separately could produce an increase in quality for the baseline algorithm. The main goal of this work is a development of several modifications of LambdaMART algorithm that are based on an optimization of gradient boosting step size for different queries. A task of testing of proposed modifications on different learning to rank datasets was set for evaluation of their performance. Experiments have shown that the obtained ranking function brings a statistically significant improvement of the base algorithm for different datasets.