Multi-armed Bandits for Personalized Recommendations

With development of modern technologies, the problem of personalizing content for each user or group of users is becoming urgent. Methods for solving the problem of multi-armed bandit for recommendations only begin to gain popularity, but even now experiments show the ability of these methods to solve this problem. However, to produce relevant personalized content, it is crucial to use additional information about users and products. This data is called a context. The presence of context leads us to the problem of a contextual multi-armed bandit. Methods for solving this problem are being actively developed. There are several approaches to this. In most cases, each context is considered isolated from the others. Then an algorithm that solves the classical task of the multi-armed bandit (MAB) is applied separately. Therefore, the problem of transferring knowledge and information between contexts becomes urgent. One way to solve this problem is to use neural networks that find nonlinear context dependencies and reward values. In this paper, we consider and propose a method based on Bayesian neural networks, in which distributions are used as weights (parameters) of the network. Neural networks allow to transfer knowledge between contexts, and distributions as weights of the neural network allow a correct exploration of the environment to balance the exploration-exploitation dilemma.