Lesson 1

Transition matrices constitute a special case of a more general framework for describing transitions between states governed by probabilistic rules. A sequence of random variables \(\alpha_{0}, \alpha_{1}, …\) is called a simple Markov chain if it satisfies, for all \(n \in \mathbb{Z}_{\ge 0}\), \[\Pr(\alpha_{n+1}= a \mid \alpha_{n} = a_{n}, \alpha_{n-1}= a_{n-1}, \dotsc, \alpha_{0} = a_{0}) = \Pr(\alpha_{n+1}= a \mid \alpha_{n} = a_{n}).\] Despite its formal mathematical formulation, the definition has a transparent intuitive meaning: a Markov chain is a sequence of states in which the state at the next step depends only on the current state and not on the entire history of previous states.