[Podcast] Neural Thickets скачать в хорошем качестве

[Podcast] Neural Thickets

https://arxiv.org/pdf/2603.12228 Neural Thickets: Post-Training LLMs via Random Weight Guessing This research introduces the concept of neural thickets, revealing that large, well-pretrained models are surrounded by a high density of task-specific experts in their local weight space. While small models require structured optimization like gradient descent to find solutions, larger models scale into a regime where randomly sampling nearby weight perturbations can effectively discover high-performing specialists. The authors propose RandOpt, a parallel post-training method that samples random perturbations, selects the best performers, and ensembles their predictions to achieve results competitive with traditional reinforcement learning. Their findings show that these sampled solutions are often diverse specialists rather than generalists, excelling at specific tasks like math or coding while potentially regressing in others. This scaling law suggests that pretraining reshapes the loss landscape into a "thicket" of accessible solutions, making downstream adaptation significantly easier. Ultimately, the study suggests that large-scale pretraining inherently prepares models for rapid, derivative-free adaptation through simple selection and aggregation. #ai #research