Beyond the Hype: How Does RAGAS Measure Real LLM Impact? скачать в хорошем качестве

Beyond the Hype: How Does RAGAS Measure Real LLM Impact?

In the rapidly evolving landscape of AI, separating genuine advancements from the noise is crucial for both industrial practitioners and academic researchers. The non-deterministic nature of LLM-driven applications amplifies this challenge, leading to a difficulty to apply common software testing practices and thereby quantify the application performance. This session by Konstantinos Sidiropoulos Software Engineer, Test and Infrastructure Capability Lead, Agile Actors and Athanasios Papaioannou AI Engineer, Agile Actors aims to offer some insight into how to mitigate these issues, by delving into RAGAS (Real-World Application and Generalization Assessment Score), a novel framework designed to quantitatively measure the performance of LLM-driven applications. Drawing from well-established practices of the software testing world, it is shown how RAGAS can be used to guide the development and deployment of LLM technologies with tangible benefits. A short demo is also displayed as a practical demonstration of how RAGAS's ability to evaluate LLM effectiveness can be translated into real world impact.