AI Sanity Checks: A Neuroscientist's Guide to Unit Testing LLMs - Hadas Baumer скачать в хорошем качестве

AI Sanity Checks: A Neuroscientist's Guide to Unit Testing LLMs - Hadas Baumer

Women on Stage Academy 2025 Graduation: AI Engineering Conference About the speaker: https://www.womenonstage.net/speakers... You just finished developing a cool new AI feature, and everybody’s excited to ship it to production. But… how can we know our LLM is doing what we think it should do? Testing a few examples looks fine, but is it enough? We wouldn't merge a pull request without passing unit tests, so why are we deploying LLMs based on our gut feeling? Before I was a Data Scientist, I was a neuroscientist, and I learned the hard way that a successful experiment comes only after a thoughtful, proactive search for faults and possible failures. I strongly feel that this mindset should be adapted into the tech world, and even more so as we advance into the new AI era, which constantly surfaces new challenges. In this talk, I will share with you a practical framework for "unit testing" your LLM. I'll show how to move from vague business goals to a concrete set of evaluation probes by deconstructing any task into its fundamental capabilities. You'll learn how to derive targeted scenarios that test these capabilities, ensuring your validation set has maximum coverage and relevance, so you’ll be able to deploy your LLMs with confidence. Enroll for the Women on Stage Academy → https://www.womenonstage.net/academy Join our global community → https://www.womenonstage.net/community Listen to our podcast → https://www.womenonstage.net/podcast Visit our Speaker Platform → https://www.womenonstage.net/speakers... Subscribe to our YouTube channel and follow us on social media: LinkedIn →   / womenonstagenet   Twitter →   / womenonstagenet   Facebook →   / womenonstagenet