Using Text Features in the Article Recommender System to Improve Key Performance Indicators

The use of sequential signals and contextual information has become a key issue of many modern recommender systems. Previous methods either employ recurrent neural networks and Transformers to capture sequential information or use different models to process external contextual information. We propose a model architecture based on the bidirectional Transformer encoder model, that will utilize both sequential and contextual information. Concretely, we use embeddings of the articles from collaborative and content-based models in the embedding layer of the modified BERT4Rec model. We build a dataset based on 7 billion interactions of users with articles and train two models – BERT4Rec and Behavior Sequence Transformer. We show that our modified version outperforms previous model that doesn’t use sequential information.