From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty

What do LLMs do when they are uncertain? We found that the stronger the LLM, the more it hallucinates and the less it loops! This pattern extends to sampling methods and instruction tuning.

We categorize fallback behaviors into types: sequence repetitions, degenerate text, and hallucinations. By pushing models towards uncertainty, we analyze their emergence across different model sizes, architectures, pretraining token counts, and instruction-following training.

We find that the more advanced an LLM is (more parameters, longer pretraining, or instruction-tuning), the more complex its fallback behaviors, shifting from sequence repetitions to degenerate text and then to hallucinations.

Even the best-performing models show this order within a single generation. As they try to recall more facts about a topic, they move from generating hallucinations to degenerate text, then to sequence repetitions.

Interestingly, common decoding techniques like random temperature sampling can reduce some behaviors (like sequence repetitions) but increase harder-to-detect hallucinations.

We also find evidence that this shift is continuous, with models becoming more degenerate with generation length, as measured by proportion of unique tokens in the sequence and compared to human baseline on the same topics.