Last decade has seen tremendous advancements in non-asymptotic analysis of system identification and optimal control for linear time-invariant (LTI) dynamical systems (e.g., [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}). Techniques and analysis developed in this paper have been initially motivated by non-asymptotic analysis of average reward-based optimal control of linear dynamical system (LDS) and switched linear dynamical system (SLDS) where expected value of the reward w.r.t stationary distribution of a Markov chain is approximated, with high probability, by its empirical averages. Although sample complexity for control/dynamical systems on continuous state space has been extremely popular recently, we still lack fundamental understanding of the factors leading to sharp concentration inequalities. For example, every ergodic Markov chain is mixing (correlation decreases asymptotically), but as we will see in the following sections that does not necessarily imply exponential concentration (especially from a single trajectory). To begin with, concentration with respect to (w.r.t) which class of functions? To expound upon these questions, in this work we leverage upon concentration of measure phenomenon for process level law of a Markov chain via transport-entropy inequality (T-E) inequalities (for a good monograph on this topic see e.g., [8]}, [9]}). The T-E approach is compact, concise and clear; even leading to exponential concentration for unbounded reward function of the form \(r(x):= \Vert x\Vert \)  for dynamical systems.
