Machine Learning Algorithms Optimization Technique for Developing Recommender Systems Using Big Data

The constant increase in the generated data provides tremendous opportunities for business. However, Big Data analysis is becoming problematic: existing systems suffer from inefficiencies in processing and analysis of large-scale data. This also affects recommender systems, which are used as a tool to provide recommendations to users. In fact, the available algorithms cannot deal with the increase in the size, variety and frequency of information alteration. That is the reason, why there is a request for a solution, which could process normalized data volumes without requiring fast CPU and spending less RAM. In the Bachelor Thesis a technique for optimizing machine learning algorithms, based on the principles of factorized machine learning, is developed and implemented. The factorized machine learning is the application of machine learning algorithms to normalized data (lying in several tables referencing each other), without performing a heavy join operation of tables into a single denormalized sample. In addition, there were done a critical analysis and systematization of existing approaches that help in working with large data. Besides, the optimized linear operations for writing and editing the code that underlies the practical part of this work were analyzed. With the help of the considered optimization technique, an increase in the speed of operation of four basic machine learning algorithms and rewritable linear operations is estimated. Finally, factorized machine learning is used to build a recommender system using a normalized set of data on viewed ads on the Avito website.