Ensuring LLM Safety | Lunchtime BABLing 60 скачать в хорошем качестве

Ensuring LLM Safety | Lunchtime BABLing 60

🚀 Subscribe to our courses: https://courses.babl.ai/p/the-algorit... 👉 Lunchtime BABLing listeners can save 20% on all BABL AI online courses using coupon code "BABLING20". 📚 Sign up for our courses today: https://babl.ai/courses/ 🔗 Follow us for more: https://linktr.ee/babl.ai 🎙️ Ensuring LLM Safety 🎙️ In this episode of Lunchtime BABLing, BABL AI CEO Dr. Shea Brown dives deep into one of the most pressing questions in AI governance today: how do we ensure the safety of Large Language Models (LLMs)? With new regulations like the EU AI Act, Colorado’s AI law, and emerging state-level requirements in places like California and New York, organizations developing or deploying LLM-powered systems face increasing pressure to evaluate risk, ensure compliance, and document everything. 🎯 What you'll learn: Why evaluations are essential for mitigating risk and supporting compliance How to adopt a socio-technical mindset and think in terms of parameter spaces What auditors (like BABL AI) look for when assessing LLM-powered systems A practical, first-principles approach to building and documenting LLM test suites How to connect risk assessments to specific LLM behaviors and evaluations The importance of contextualizing evaluations to your use case—not just relying on generic benchmarks Shea also introduces BABL AI’s CIDA framework (Context, Input, Decision, Action) and shows how it forms the foundation for meaningful risk analysis and test coverage. Whether you're an AI developer, auditor, policymaker, or just trying to keep up with fast-moving AI regulations, this episode is packed with insights you can use right now. 📌 Don’t wait for a perfect standard to tell you what to do—learn how to build a solid, use-case-driven evaluation strategy today. 👍 Like this video? Subscribe and hit the bell for more episodes exploring the intersection of AI, ethics, law, and governance. 👉 TIMESTAMPS 00:00 – Intro: Why LLM Evaluations Matter for Risk & Compliance 00:55 – Overview: Ethics, Risk & Regulatory Pressures (EU AI Act, Colorado, NY) 01:32 – Key Takeaways: Evaluations, Sociotechnical Mindset & Documentation 02:31 – Why You Can’t Wait for Standards—Focus on First Principles 04:02 – Regulatory Pressure: EU AI Act Article 9 & NX3 Obligations 05:30 – Why Evaluations Are Essential for AI Systems 07:33 – A Basic Framework for LLM Testing & Documentation 08:30 – From an Auditor’s Perspective: What You Need to Prove 09:18 – What to Document: Context, Users, Use Cases & Fail States 10:19 – Introducing the CIDA Narrative: Context → Input → Decision → Action 12:19 – How to Run a Risk Assessment for LLMs 13:07 – Common LLM Risks: Confabulations, Toxicity & Robustness 15:51 – Grouping Risks: Using NIST & Custom Categories 18:14 – Using HELM Benchmarks and When to Customize Tests 19:30 – Prompt/Response Testing & Quantifying Performance 21:44 – Test Coverage Strategy: Focus on Both Risk & Performance 22:24 – Parameter Space Thinking: Mapping Real-World Complexity 24:28 – How to Probe the AI System’s Full Behavioral Landscape 26:08 – Capturing the Full Chain: From Inputs to Consequences 27:42 – Outro: Subscribe for More on AI Risk, Governance & Testing #ResponsibleAI #LLMSafety #AIAudit #EUAIACT #LunchtimeBABLing #AIEthics #BABLAI #AI #Compliance