Generative Models in Machine Learning
- The learning objective of the course “Generative methods in Machine Learning” is to provide students advanced techniques and deeper theoretical and practical knowledge in modern probabilistic learning techniques, such as: <ul> <li> Basic principles, Generative Models,</li> <li> Bayesian Network, Random Markov Fields, Boltzmann Machines, Variational Auto Encoders</li> <li> Sampling and Inference, Variational inference, variational methods, </li> <li> Neural Networks,</li> <li> Deep Learning techniques.</li> </ul>
- Students are introduced to probabilistic modelling of data.
- Student are introduced to generalization and theory of the basic methods.
- Students are introduced to generalization and theory of the basic methods.
- Students are introduced to generating new data from probabilistic distributions.
- Students know variational methods for learning to represent data distribution.
- Students know state-of-the-art methods currently used in data generation processes.
- Students are introduced to deep generative models such as deep belief networks, etc.
- Introduction to machine learning, Bayesian Decision Theory, Maximum Likelihood Estimation, and EMBasic definitions, principles and types of machine learning. Classifiers, Discriminant Functions, and Decision Surfaces, Minimum-Error-Rate Classification, Neyman-Pearson lemma, Distributions, Rela-tion to Logistic Regression, Naïve Bayes classification, basics of MLE, learning parameters of distri-butions. Gaussian Mixture Models, Latent Variables, Examples, Expectation-Maximization, Latent Dirichlet Allocation.
- Exponential Family, Sufficient StatisticsGeneralized Linear Models.
- Graphical Models and Generative LearningBayesian Networks, Random Markov Fields, Conditional Random Fields, Boltzmann Machines, Energy-based methods. Hidden Markov Models.
- Sampling and InferenceExact and Inexact Inference, Gibbs sampling, Bridge Sampling, Simple and Annealed Importance Sampling, Monte-Carlo EM, Junction Tree algorithm.
- Variational LearningMean-Field, Bethe Approximation, Variational methods, Variational Message Passing, Free-Energy, Variational Free Energy. Variational Bayes, Variational Bayes Expectation-Maximization. Mean field methods.
- Generative ModelsRestricted Boltzmann Machines, Helmoltz Machines and Wake-Sleep algorithms, Energy-based methods. Generative Adversarial Networks, Generative Auto-Encoders, Belief networks, connectionist learning. Variational AutoEncoders.
- Deep learning techniquesNeural Networks, Shallow networks, Multilayer Neural networks, back-propagation, deep learning, Universal Approximation. Auto Encoders, Stacked Auto-Encoders, Stacked Boltzmann machines, supervised and unsupervised pre-training, Deep Belief Networks. Deep Universality theorems.
- Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Boston: Now Publishers. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=nlebk&AN=352768
- James, G. et al. An introduction to statistical learning. – Springer, 2013. – 426 pp.