Back to the Future – Data Efficient Language Modeling with Tatsunori Hashimoto скачать в хорошем качестве

Back to the Future – Data Efficient Language Modeling with Tatsunori Hashimoto

Tatsunori Hashimoto from Stanford University visited the Kempner Seminar Series on October 3, 2025, to discuss "Back to the Future – Data Efficient Language Modeling." Compute scaling has dominated the conversation with modern language models, leading to an impressive array of algorithms that optimize performance for a given training (and sometimes inference) compute budget. But as compute has grown cheaper and more abundant, data is starting to become a bottleneck, and our ability to exchange compute for data efficiency may be crucial to future model scaling. In this talk, I’ll discuss synthetic data and algorithmic approaches to data efficiency, and show that in both cases, classical statistical perspectives based on nonparametric modeling and ensembling bring new insights and significant empirical benefits to modern questions of scaling and data efficiency.