On the Role of Temperature Sampling in Test-Time Scaling скачать в хорошем качестве

On the Role of Temperature Sampling in Test-Time Scaling

Paper: https://arxiv.org/abs/2510.02611 Title: On the Role of Temperature Sampling in Test-Time Scaling Authors: Yuheng Wu, Azalia Mirhoseini, Thierry Tambe Abstract: Large language models (LLMs) can improve reasoning at inference time through test-time scaling (TTS), where multiple reasoning traces are generated and the best one is selected. Prior work shows that increasing the number of samples K steadily improves accuracy. In this paper, we demonstrate that this trend does not hold indefinitely: at large K, further scaling yields no gains, and certain hard questions remain unsolved regardless of the number of traces. Interestingly, we find that different sampling temperatures solve different subsets of problems, implying that single-temperature scaling explores only part of a model's potential. We therefore propose scaling along the temperature dimension, which enlarges the reasoning boundary of LLMs. Averaged over Qwen3 (0.6B, 1.7B, 4B, 8B) and five representative reasoning benchmarks (AIME 2024/2025, MATH500, LiveCodeBench, Hi-ToM), temperature scaling yields an additional 7.3 points over single-temperature TTS. Temperature scaling also enables base models to reach performance comparable to reinforcement learning (RL)-trained counterparts, without additional post-training. We further provide a comprehensive analysis of this phenomenon and design a multi-temperature voting method that reduces the overhead of temperature scaling. Overall, our findings suggest that TTS is more powerful than previously thought, and that temperature scaling offers a simple and effective way to unlock the latent potential of base models. Tags: Machine Learning, Research, Mathematics, reinforcement learning, gan, transformer, attention, self-attention, search, role, temperature, sampling, test, time, research paper, academic, study, analysis, tutorial, explained, breakdown, paper review, research summary, AI research, scientific paper, methodology, results, findings, innovation, technology, computing, algorithm, model, dataset, evaluation, performance, accuracy, efficiency, optimization, deep learning Welcome to the Mayuresh Shilotri's Youtube . Maintained by Mayuresh Shilotri You can follow me at Blog - https://shilotri.com/ LinkedIn -   / mayureshshilotri   Twitter -   / mshilotri   Note: I only claim to have read the research paper and created a Video using AI tool. I am not the author. All intellectual heavy lifting was performed by the respective authors. 🙏