One Sentence to Rule Them All: How to Mitigate Mode Collapse and Unlock LLM Diversity скачать в хорошем качестве

One Sentence to Rule Them All: How to Mitigate Mode Collapse and Unlock LLM Diversity

Large Language Models (LLMs), particularly after standard alignment methods, often experience a loss of output variety known as "mode collapse," causing them to favor a narrow set of typical responses and hindering performance in creative and open-ended tasks. This paper identifies that this diversity reduction stems primarily from a *pervasive data-level driver* called **typicality bias**, where human annotators systematically favor familiar or conventional text when providing preference data, thereby training the LLMs to narrow their responses to stereotypical choices. To counteract this bias, the authors introduce **Verbalized Sampling (VS)**, a simple, training-free prompting strategy that works during inference by asking the model to explicitly generate a set of potential responses along with their estimated probabilities (e.g., “Generate 5 jokes about coffee and their corresponding probabilities”). By requesting this distribution, VS compels the model to approximate the broader, more diverse distribution it learned during initial pre-training, successfully recovering the LLM's inherent generative diversity. Extensive experiments across tasks like creative writing, dialogue simulation, and synthetic data generation show that VS substantially enhances output diversity—for example, increasing diversity in creative writing by 1.6 to 2.1 times over direct prompting—without sacrificing the model's factual accuracy or safety. https://arxiv.org/pdf/2510.01171 https://github.com/CHATS-lab/verbaliz...