The Art of Being Wrong: How to Calibrate Confident AI Models That Lie скачать в хорошем качестве

The Art of Being Wrong: How to Calibrate Confident AI Models That Lie

Most product teams spend a whole quarter trying to fix a model that will never be perfect. Pranav Pathak spent his early career doing exactly that, until he realized the model wasn't the problem. In this session, Nicole Gottselig sits down with Pranav Pathak, Product Director for AI & ML at Booking.com, for a live breakdown of why confident AI fails in the real world and what product teams can actually do about it. Pranav walks through real lessons from building AI at scale, including a story from nine years ago where he couldn't get a spam model above 90% accuracy and had to stop trying to fix the ML and start fixing the UX instead: Why "Zombie Projects" happen when teams keep throwing data and algorithms at a model instead of designing around its limits The 99/80/40 Rule in practice: why a 99% accurate cancellation model should never cancel an order, and what to do at every confidence level How 75-80% of Netflix's views come from recommendations, and why the cost of being wrong is near zero What Air Canada's chatbot lawsuit teaches us about the one thing you can never let AI make up How Booking.com lowered precision thresholds on a customer service routing model to cover more use cases and actually improved the product Using a judge LLM to catch 90-95% of inaccuracies before they reach a user Training a model to say "I don't know" and why that counts as the correct answer Why Pranav tracks AI capabilities (summarization, intent detection, generation) instead of tracking models, and how that keeps his team calm Whether you're a product manager trying to ship AI that survives month six, or someone trying to figure out why your demo felt so promising and your roadmap feels stuck, this conversation will reframe how you think about AI accuracy entirely. Timestamps 00:00 Introduction and Welcome 01:17 Why Building with AI Is a Soft Skill, Not a Science 02:30 Understanding Model Accuracy: The 99%, 80%, and 40% Rule 08:27 Practical Applications of AI in Business 14:25 Building AI with Fail-Safes and Human Oversight 20:31 The Future of Jobs in the Age of AI Connect with Pranav on LinkedIn:   / pranavmpathak