Random Variables · Theory Problems

Lesson 3

Nikolai Chukhin · Alexander S. Kulikov

Optional Problems.

  1. (20 points) A hen lays a random number of eggs \(N\), which follows a Poisson distribution with parameter \(\lambda\). Each egg hatches independently with probability \(p\). Let \(K\) be the number of chicks that hatch. What is the value of \(E[K]\)?
    Hint:
    Consider representing \(K\) as a sum of indicator random variables \(X_{i}\), where \(X_{i}\) takes the value \(1\) if the \(i\)-th egg hatches and \(0\) otherwise. Remember that the limit of the summation \(N\) is itself a random variable.
  2. (20 points) Let \(X_{1}, X_{2}, …, X_{n}\) be independent and identically distributed (i.i.d.) random variables with finite expectation. Let \(S_{n} = X_{1} + … + X_{n}\). Prove that \(E[X_{1} \mid S_{n}] = \frac{S_n}{n}\).
  3. (20 points) A miner is trapped in a mine containing \(3\) doors.
    • Door 1 leads to a tunnel that will take him to safety after \(3\) hours.

    • Door 2 leads to a tunnel that returns him to the mine after \(5\) hours.

    • Door 3 leads to a tunnel that returns him to the mine after \(7\) hours.

    The miner chooses a door with equal probability. If he returns to the mine, he chooses again with the same probabilities. Let \(T\) be the total time to escape, and find \(E[T]\).
  4. (20 points) Let \(X\) be a random variable uniformly distributed on the set \(\{1, 2, 3, 4\}\). Let \(Y\) be a random variable that, given \(X=x\), is uniformly distributed on the set \(\{1, 2, …, x\}\). We want to predict \(X\) based on \(Y\) to minimize the mean squared error. Find the estimator \(f(Y) = E[X|Y]\) and explicitly calculate its value for \(Y=1\).
  5. (20 points) Suppose \(E[\alpha^{2}]<\infty\). Let \(H\) be the space of all functions of a random variable \(\beta\) with finite expectation of their square (i.e. if \(f \in H\), then \(E[f(\beta)^{2}]\) is finite). The conditional expectation \(\hat{\alpha}= E[\alpha \mid \beta]\) can be viewed as the projection of \(\alpha\) onto \(H\). Prove the Orthogonality Principle: For any function \(g(\beta)\in H\), the prediction error \((\alpha - E[\alpha \mid \beta])\) is orthogonal to \(g(\beta)\), i.e., \[E[(\alpha - E[\alpha \mid \beta]) \cdot g(\beta)] = 0.\] Then, explain how this implies that \(E[\alpha \mid \beta]\) minimizes the Mean Square Error.
    Hint:
    To prove the orthogonality, apply the Wald's identity to the term \(E[\alpha \cdot g(\beta)]\). Remember that if \(g(\beta)\) is a function of \(\beta\), it can be pulled out of the conditional expectation \(E[\cdot \mid \beta]\).

    To show that \(\hat{\alpha}\) minimizes the MSE, consider an arbitrary estimator \(h(\beta) \in H\). Expand the term \(E[(\alpha - h(\beta))^{2}]\) by adding and subtracting \(E[\alpha \mid \beta]\) inside the squared expression, and use the orthogonality result to eliminate the cross-term.