Top-k / Top-p Sampling

Friendly Description: Top-k and top-p are two ways an AI decides which word to say next. Top-k means "only consider the k most likely options." Top-p (also called nucleus sampling) means "only consider enough options to cover p% of the probability." Both are ways of keeping responses interesting without letting the AI go off the rails into nonsense.

Example: With top-k of 5, when picking the next word the model only chooses among its 5 best guesses, ignoring the long tail of unlikely options. With top-p of 0.9, the model only considers as many top options as needed to cover 90% of the probability mass. Together with temperature, these settings shape how creative or focused the output feels.