Simulating & Evaluating Multi turn Conversations скачать в хорошем качестве

Simulating & Evaluating Multi turn Conversations

Most LLM applications today are chat-based. How would you evaluate the conversations? One way to evaluate is to create a simulation with another AI — where one model plays the role of the user to interact with your chat-based application, and you observe how the conversation unfolds over multiple turns to test for metrics such as helpfulness, consistency, and goal completion. We’re excited to launch OpenEvals — a set of utilities to simulate full conversations and evaluate your LLM application’s performance. OpenEvals: https://github.com/langchain-ai/opene... Notebook: https://github.com/catherine-langchai... 0:00 Introduction 0:40 Overview of Multi-Turn Simulation in OpenEvals 2:34 Example of Evaluating a Deployed Agent over Multi-Turn Conversation