Lesson 1

In applications, we often do not know the distribution of a random variable. Instead, we have observations: \[x_{1},x_{2},\dotsc,x_{N}.\] For example, these could be daily numbers of requests to a server or lengths of messages sent by users. The simplest way to turn these observations into a probability model is to use the empirical distribution. It is the distribution of the following experiment:

In other words, the empirical distribution does not guess what values could have occurred but did not occur. It puts all its probability mass on the data we actually saw. For any set \(A \subseteq \mathbb{R}\), define \[\widehat{\Pr}_{N}[\alpha \in A] =\frac{1}{N}\sum_{i=1}^{N}[x_{i} \in A].\] I.e., the empirical distribution assigns to a set the fraction of observed values that lie in it.