Lesson 2

Consider a simple example. Suppose you randomly draw balls of different colors from a bag. If the drawn balls are not returned to the bag, then the probability of drawing a ball of a particular color depends on all previously drawn balls. Consequently, in order to determine this probability, one must know the entire history of past outcomes. Such a process therefore does not satisfy the Markov property and cannot be described as a Markov chain. Now assume instead that we initially select a ball at random and, at each step, replace it with a randomly chosen ball from the bag (that is, we draw a new ball from the bag and then return the previously selected ball). In this case, the probability of drawing a ball of a given color at the next step depends only on the current state of the system (namely, the ball currently in hand) and not on earlier draws. Hence, this process satisfies the Markov property and can be modeled as a Markov chain.