Lesson 3

In classification problems, one random variable is usually a vector of observed features and another is the hidden label. We may denote them by \(X\) and \(Y\). A probabilistic classifier tries to estimate \[\Pr[Y=y \mid X=x].\] After that, the most natural prediction is the label with the largest conditional probability.

The same language describes text prediction. A language model can be viewed as producing a conditional distribution \[\Pr[\text{next token}=w \mid \text{previous tokens}].\] The model is not merely outputting a word; it is outputting a probability distribution over possible next words. This makes it possible to ask more precise questions: which continuation has the largest probability, how much probability mass is spread over many alternatives, or how likely it is that the generated continuation has a specified property. Sampling from this distribution is what makes two runs of the same prompt able to produce different continuations.