Modern Methods of Decision Making
- Knowledge of classical learning algorithms and their performance
- Understand the basic tradeoff between model complexity and computational tractability
- Understand introduction to learning theory.
- Understand and use optimal predictors.
- Understand learning with finite classes and know how to use it.
- Understand the learning with complex models and know how to use it.
- Understand learning through optimisation and know how to use it.
- Understand algorithmic interlude and know how to use it.
- Understand and use online learning.
- Introduction to learning theoryIn this chapter, we introduce the basic language of supervised learning theory and describe the notions of: learning sample, loss function, risk, optimal/Bayes predictors and excess risk. In particular, we define the PAC learning paradigm (Probably Approximately Correct)
- Optimal predictorsWe provide a full description of optimal (or ideal) predictors associated to a given loss function.
- Learning with finite classesWe study the ERM (Empirical Risk Minimisation) principle in the simple context of a model composed of a finite number of candidate predictors.
- Learning with complex modelsWe study a generalisation of the results seen in the previous chapter for models composed of potentially an infinite number of candidate predictors. We introduce the notion of VC dimension of a model and prove the fundamental theorem of learning theory.
- Learning through optimisationWe study classical algorithms from convex optimisation to compute empirical risk minimisers in practice.
- Algorithmic interludeWe explore several algorithms, related to ERM, such as Boosting, SVM's.
- Online learningWe provide a first look at the world of online learning. We focus on the framework of prediction with expert advice with full and limited feedback. We will provide in particular a full analysis of the EWA and EXP4.P algorithms.
- Interim assessment (4 module)0.2 * Final exam + 0.4 * Home assignment 1 + 0.4 * Home assignment 2
- Mohri, M., Talwalkar, A., & Rostamizadeh, A. (2012). Foundations of Machine Learning. Cambridge, MA: The MIT Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=478737
- Christopher M. Bishop. (n.d.). Australian National University Pattern Recognition and Machine Learning. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.EBA0C705