- Learning the audience what are the Markov chains with finite number of states and the corresponding basic technique.
- Learning possible types of large-time behavior of the Markov chains with finite number of states.
- Learning some applications of the Markov chains technique to various examples arising in different areas.
- The students are expected to be familiar with a number of examples of Markov chains with finite number of states and to be able to recognize if a given discrete stochastic system with finite number of states is a Markov chain or not and apply the Markov chain technique to study this system if the system forms a Markov chain.
- After the course the students are expected to understand what is a Markov chain with finite number of states, to know its basic properties, possible types of its large-time behavior and possible topological structure.
- Markov chains with finite number of states
- Stationary states and their existence
- Ergodic theorem for Markov chains with ergodic transition probability matrix
- Applications of the ergodic theorem. The law of large numbers for Markov chains. The Google’s Page Rank.
- Perron–Frobenius theorem
- Topological structure of Markov Chains
- Periodic Markov chains.
- Aperiodic Markov chains. Ergodic theorem for irreducible aperiodic Markov chains
- Gnedenko, B. V., & Ushakov, I. A. (1997). Theory of Probability (Vol. Sixth edition). Amsterdam, the Netherlands: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1835590
- Meyn, S. P., & Tweedie, R. L. (2009). Markov Chains and Stochastic Stability (Vol. 2nd ed). Cambridge: Cambridge University Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=313161
- Ibe, O. C. (2013). Markov Processes for Stochastic Modeling (Vol. 2nd edition). Chennai: Elsevier. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=516132