Why LLM Judges Suck, in Less Than 5 Minutes скачать в хорошем качестве

Why LLM Judges Suck, in Less Than 5 Minutes

LLM Judges are totally overhyped. Find out why and what they're actually good for from Doug Turnbull, who led Search at Reddit + Shopify. This is a 5 minute distillation of Doug's blog post "LLM Judges aren’t the shortcut you think": https://softwaredoug.com/blog/2025/11... Join us for an upcoming livestream "A Builder’s Guide to Agentic Search & Retrieval "with Doug Turnbull & John Berryman (register to join live or get the recording afterwards): https://luma.com/6os4cy5g lu.ma calendar: https://luma.com/calendar/cal-8ImWFDQ... Want to learn more? Join the final cohort of our Building AI Applications course in Q1, 2026 (25% off for viewers): https://maven.com/hugo-stefan/buildin... 00:00 Introduction to LLM Judges 00:27 Problem 1: Lack of User Engagement Understanding 01:08 Problem 2: Importance of the Last 10% 01:47 Problem 3: Sneaky Overfitting 02:17 Problem 4: Limited Document Evaluation 02:49 Problem 5: Trust Issues with Novel Use Cases 03:09 Effective Use of LLM Judges