LLM Evaluation and Testing for Reliable AI Apps - MLOps Live #38 with Evidently AI скачать в хорошем качестве

LLM Evaluation and Testing for Reliable AI Apps - MLOps Live #38 with Evidently AI

In this webinar, we heard firsthand about the challenges and opportunities presented by LLM observability. We discussed: -Real-world risks: Shared actual examples of LLM failures in production environments, including hallucinations and vulnerabilities. -Practical evaluation techniques: Discovered tips for synthetic data generation, building representative test datasets, and leveraging LLM-as-a-judge methods. -Evaluation-driven workflows: Explored how to integrate evaluation into your LLM product development and monitoring processes. Production monitoring strategies: Gain insights on adding model monitoring capabilities to deployed LLMs, both in the cloud and on-premises. Relevant Links: 1. LLM monitoring in MLRun: https://docs.mlrun.org/en/latest/tuto... 2. Monitoring in MLRun with the Evidently base class: https://docs.mlrun.org/en/latest/api/...[…]identlyModelMonitoringApplicationBase