Master
2020/2021

## Applied Machine Learning

Type:
Elective course (Big Data Systems)

Area of studies:
Business Informatics

Delivered by:
Department of Business Informatics

When:
1 year, 3 module

Mode of studies:
distance learning

Open to:
students of all HSE University campuses

Instructors:
Sergey Lisitsyn

Master’s programme:
Big Data Systems

Language:
English

ECTS credits:
5

### Course Syllabus

#### Abstract

Machine learning is the field of study that helps us to find the dependencies in data automatically. Such a technology enables to solve different problems without explicit programming of rules. Due to advances in computing and the field itself, during last decade machine learning has become an essential feature of products ranging from web-services to banks. In this course the student is going to overview the essential concepts of machine learning and then practice employing machine learning methods to solve business tasks. This course emphasizes the practical part and considers various aspects of solving real-world problems. The course content covers all the popular methods such as linear methods, gradient boosting, and neural networks. Finally, the course considers the best practices of major companies leveraging the machine learning technology.

#### Learning Objectives

- Learn to identify a machine learning problem to solve a business problem
- Practice fitting models to solve essential machine learning problems such as regression and classification
- Learn to design and to develop machine learning systems
- Learn to re-use pre-trained models to lower the development cost of a machine learning systems

#### Expected Learning Outcomes

- Can identify a problem suitable for machine learning
- Knows at least a few modern applications of machine learning
- Able to identify classification, regression, and clustering problems
- Knows the limitations of linear models
- Able to fit a logistic regression model on a given dataset
- Able to fit and interpret a decision tree model on a given dataset
- Able to apply gradient boosting approach to solve classification and regression problems
- Able to identify overfitting
- Knows the relations between complexity and overfitting
- Understands the boosting approach to create an ensemble of models
- Understands the universality of gradient boosting approach
- Can identify a recommender problem
- Understands the essential methods for recommenders: collaborative filtering, content-based, and matrix factorization
- Understands the concept of embeddings
- Understands the concept of non-parametric learning
- Able to identify a clustering problem
- Can fit a clustering model given a dataset
- Able to identify the suitable metric for a machine learning system
- Understands the concept of differentiable programming
- Able to train a neural network given a dataset
- Understands the idea of convolution as the base operation for images and audio data
- Able to use pre-trained models
- Knows the essential rules to develop and support machine learning systems

#### Course Contents

- Scope of machine learningReasons to use machine learning and the limitations of current technologies. An overview of applications of machine learning.
- Machine learning problemsClassification, regression, and clustering.
- Linear models for regression and classificationThe formal definition. The logistic regression model. Loss functions and regularizers.
- Decision trees and ensemblesDecision tree models. Entropy and Gini as measures of information for a split. Learning algorithms: ID3 and CART.
- OverfittingThe overfitting effect and it's causes. Ways to solve overftting. Ensembles of various models.
- Boosting and gradient boostingThe approach of boosting to increase the complexity of models. Overfitting and boosting. Relations of boosting and gradient-based methods. The gradient boosting approach.
- Recommender systems and embeddingsCollaborative filtering and content-based recommenders. The matrix factorization approach. Relations between matrix factorization and embeddings. Embeddings as the modern way to solve recommendation problems.
- Non-parametric methods for classification and regressionThe k-nearest neighbor algorithm. Kernel variant of Support Vector Machine method. Gaussian processes.
- ClusteringClustering as ill-posed problem. The k-means algorithm. Other approaches such as hierarchical clustering and DBSCAN.
- Metrics of machine learningThe role of metrics and losses in machine learning. Regression metrics such as MSE, RMSE, and MAE. Classification metrics such as precision, recall, and accuracy. Estimating quality of clustering.
- Neural networksThe model of neuron. Neural networks as complex differentiable functions. The gradient descent and chain differentiation as the backpropagation algorithm. Differentiable programming.
- Convolutional neural networksConvolution as the image processing operation. The role of convolutions in feature detection. Convolutional neural layer. Modern architectures of convolutional neural networks.
- Machine learning in production systemsEssential rules of machine learning in production systems. The best practices from companies using machine learning.

#### Assessment Elements

- Homework №1A student should provide a Jupyter notebook.
- Homework №2A student should either provide a Jupyter notebook to the professor, or participate in an in-class Kaggle competition.
- MOOCA student is graded in a binary way based on the completion of the MOOC.
- Written examFormat: the exam is taken written (programming assignment). Students are not allowed to involve any other person in their programming assignment. Any interaction with other students that gives advantage on the assignment is prohibited and so is any plagiarism in the programming assignment. Students are allowed to use any Internet resources and clarify their assignment with the professor.

#### Interim Assessment

- Interim assessment (3 module)0.3 * Homework №1 + 0.3 * Homework №2 + 0.2 * MOOC + 0.2 * Written exam

#### Bibliography

#### Recommended Core Bibliography

- D. Sculley, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, & Michael Young. (n.d.). Machine Learning: The High-Interest Credit Card of Technical Debt. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.BAEF1F2C
- Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning : Data Mining, Inference, and Prediction (Vol. Second edition, corrected 7th printing). New York: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=277008
- Segaran, T. (2007). Programming Collective Intelligence : Building Smart Web 2.0 Applications. Beijing: O’Reilly Media. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=415280

#### Recommended Additional Bibliography

- Caselles-Dupré, H., Lesaint, F., & Royo-Letelier, J. (2018). Word2Vec applied to Recommendation: Hyperparameters Matter. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsarx&AN=edsarx.1804.04212