• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Controllable Markov Chain for Analysis of Reliability Models and Problems of Dynamic Managment

Student: Yarullina Liya

Supervisor: Victor Kashtanov

Faculty: HSE Tikhonov Moscow Institute of Electronics and Mathematics (MIEM HSE)

Educational Programme: Applied Mathematics (Bachelor)

Final Grade: 9

Year of Graduation: 2017

The subject of this bachelor’s thesis is Markov chain as one of the key examples of random processes. In the course of the work, a wide range of studies and research papers, which describe Markov chains, controlled Markov and semi-Markov processes for reliability and queuing theories, were examined. The purpose of this study is to describe and analyze an abstract controlled semi-Markov process, which presents the connection between models of dynamic programming and controlled Markov processes. Moreover, it was necessary to find optimal control strategies. In the theoretical section of the thesis, a set of states and controls were described, an achievement functional was built, its structure was defined and some useful theorems for optimal strategies searching were given. In the practical section, the author modified the “transport choice” described in [11] as a problem of semi-Markov process, and then, using methods of dynamic programming, wrote a program in C++ to calculate the time and find optimal strategies. It was found that the optimal strategy has a specific "threshold" property, that is, up to a certain value one decision strategy is optimal, but after that value – the other one.

Full text (added May 25, 2017)

Student Theses at HSE must be completed in accordance with the University Rules and regulations specified by each educational programme.

Summaries of all theses must be published and made freely available on the HSE website.

The full text of a thesis can be published in open access on the HSE website only if the authoring student (copyright holder) agrees, or, if the thesis was written by a team of students, if all the co-authors (copyright holders) agree. After a thesis is published on the HSE website, it obtains the status of an online publication.

Student theses are objects of copyright and their use is subject to limitations in accordance with the Russian Federation’s law on intellectual property.

In the event that a thesis is quoted or otherwise used, reference to the author’s name and the source of quotation is required.

Search all student theses