Master
2025/2026
Markov Chains
Type:
Compulsory course (Math of Machine Learning)
Delivered by:
Big Data and Information Retrieval School
Where:
Faculty of Computer Science
When:
1 year, 2 module
Open to:
students of one campus
Language:
English
ECTS credits:
3
Contact hours:
42
Course Syllabus
Abstract
The course is an introduction to the theory of Markov chains, an area of modern probability theory widely used in applications. In this course, we will start from the theory of finite state space Markov chains and continue with the general case of Markov chains with arbitrary state space. We will cover various ergodicity results for Markov kernels and relations between them, central limit theorem for Markov chains, and applications of Markov chains. In terms of applications, we will consider the Markov Chain Monte Carlo methods and study some most classical examples of algorithms of this family.
Learning Objectives
- Purpose of course: understanding mathematical fundamentals of the Markov chains and their applications in machine learning models.
Expected Learning Outcomes
- The ability to apply Markov chains methodology to solve theoretical problems in related fields
- Ability to apply and analyze the Markov Chain Monte Carlo (MCMC) algorithms
- Developing the skill of problem-solving and understanding the theoretical concepts
Course Contents
- Finite state-space Markov Chains.
- Markov kernels. Basics of Markov Chains with general state space.
- Geometric ergodicity of general state-space markov kernels.
- Coupling methods
- Central limit theorems (CLTs) for Markov chains
- Applications: MCMC
Assessment Elements
- HomeworksTasks for the topics covered in class
- ExamThe oral exam includes the main topics covered in lectures and seminars.
Bibliography
Recommended Core Bibliography
- Markov chains, Revuz, D., 2005
Recommended Additional Bibliography
- Meyn, S. P., & Tweedie, R. L. (2009). Markov Chains and Stochastic Stability (Vol. 2nd ed). Cambridge: Cambridge University Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=313161