Annus Mirabilis: A Year of Explosive Progress in LLMs with Benjamin Feuer скачать в хорошем качестве

Annus Mirabilis: A Year of Explosive Progress in LLMs with Benjamin Feuer

Benjamin Feuer is a Postdoctoral Researcher at Stanford University working with Ludwig Schmidt's Lab. He completed my Ph.D. in Computer Science and Engineering at NYU as a member of the DICE Lab, and is also a researcher for AI startup Oumi.AI. Previously, he received an BA in Film Studies from Wesleyan University, an MFA in Screenwriting from Columbia University, and an MS in Computer Science from New York University. His awards include a NeurIPS Spotlight award and the Deborah M. Rosenthal Award (Best CS Qualifying Exam). LLMs have experienced an extended "annus mirabilis" since the public debut of ChatGPT, with innovation proceeding at an unprecedented pace. But many open scientific questions remain. When is it safe to replace human evaluators with LLM judges? What are the trade-offs we can expect? How trustworthy are the benchmarks we use to measure progress, and how can we improve them? How can we maximize the potential of synthetic data for training LLMs? In this talk, Benjamin Feuer covers some of his recent research in these areas, and highlights some areas he thinks are important for future progress.