Lesson 7

But what if we want to predict the value of a random variable \(\alpha \colon U \to \mathbb{R}\) not just like that, but based on some observations to which we have access, that is, based on another random variable \(\beta \colon U \to \mathbb{R}\)? For example, we want to learn to predict an employee's income based on their age and education. For what function \(f \colon \mathbb{R} \to \mathbb{R}\) will the mean square deviation \[\operatorname{E}\left[(\alpha - f(\beta))^{2}\right]\] be minimal? It turns out that it is minimized by the conditional expectation \(\alpha\) with respect to the random variable \(\beta\). This is the random variable \[f(\beta)=\operatorname{E}[\alpha \mid \beta] \colon U \to \mathbb{R} \ ,\] defined as: \[\operatorname{E}[\alpha \mid \beta](u)=\operatorname{E}[\alpha \mid \beta=\beta(u)] \ .\] As can be seen, this is indeed a function of the random variable \(\beta\): to each value \(b \in \mathbb{R}\) of the random variable \(\beta\), it assigns the number \(\operatorname{E}[\alpha \mid \beta=b]\).