Lesson 2

Often, different notations are used for events: \(H\) (hypothesis) and \(E\) (evidence). We assume that \(\Pr[H]>0\) and \(\Pr[E]>0\). \[\Pr[H \mid E] = \frac{\Pr[E \mid H]}{\Pr[E]}\cdot \Pr[H] \ .\] Here, \(\Pr[H]\) is called the prior probability of \(H\), and \(\Pr[H \mid E]\) is the posterior probability. In other words, the first is our estimate of the probability of hypothesis \(H\) before we learn that event \(E\) has occurred, and the second is after we learn it. Bayes' formula precisely explains how these two estimates are related. To adjust our estimate of probability \(\Pr[H]\) considering that event \(E\) has occurred, we need to multiply it by \(\frac{\Pr[E \mid H]}{\Pr[E]}\). (Note that when considering events, we do not think about which event occurred first. We don't even have the concept of time. However, here we are talking about applying probability theory in real life (that is, roughly speaking, statistics), where time does exist.)

Bayes' formula can also be found in the following form: \[\Pr[H \mid E] = \frac{\Pr[E \mid H]}{\Pr[E \mid H]\Pr[H]+\Pr[E \mid \neg H]\Pr[\neg H]}\cdot \Pr[H] \ .\] This is indeed the same Bayes' formula, where \(\Pr[E]\) is expanded using the law of total probability.