Problèmes de premier passage et de commande optimale pour des chaînes de Markov à temps discret
We consider diffusion processes, deﬁned by stochastic differential equa-  tions, and then we focus on ﬁrst passage problems for Markov chains in dis-  crete time that correspond to these diffusion processes. As it is known in the  literature, these Markov chains converge in distribution to the solution of the  stochastic differential equations considered. Our contribution is to obtain ex-  plicit formulas for the ﬁrst passage probability and the duration of the game  for the discrete-time Markov chains. We also show that the results obtained  converge in the Euclidean metric to the corresponding quantities for the diffu-  sion processes.  Finally we study an optimal control problem for Markov chains in discrete  time. The objective is to ﬁnd the value which minimizes the expected value of  a certain cost function. Unlike the continuous case, an explicit formula for this  optimal value does not exist in the discrete case. Thus we study in this thesis  some particular cases for which we found this optimal value.
Discrete-time Markov chains Fnancial mathematics Diffusion processes Absorption problems Difference equations Wiener process Special  functions Optimal control Principle of optimality