Lesson 2

Conditional distributions describe how one variable behaves after we learn something about another. In the discrete case, for every \(a\) such that \(\Pr[\alpha=a]>0\), \[\Pr[\beta=b \mid \alpha=a] = \frac{\Pr[\alpha=a,\beta=b]}{\Pr[\alpha=a]}.\] If this conditional distribution is the same for all relevant values of \(a\), then \(\alpha\) gives no information about \(\beta\). This is exactly independence.

In data analysis, this is one of the most important questions: does one quantity help predict another? For instance, does the length of an email help predict whether it is spam? Does a user's recent activity help predict whether they will click a recommendation? Does the prefix of a sentence help predict the next word?