Gao & Slater - Use ARTKIT to Automate and Scale Up Your LLM Evaluation Process | PyData NYC 2024 скачать в хорошем качестве

Gao & Slater - Use ARTKIT to Automate and Scale Up Your LLM Evaluation Process | PyData NYC 2024

www.pydata.org Since late 2022, Large Language Models (LLMs) have become an integral part of our daily lives, propelled by the rise of ChatGPT. Amid continuous media coverage, companies like Nvidia have experienced stock surges, and individuals have adjusted their work and study habits in response to these developments. However, many organizations remain hesitant to adopt LLMs and Generative AI (GenAI) at scale, primarily due to insufficient comprehensive experimentation and testing. There is a concern that deploying LLMs/GenAI without proper alignment to business needs could incur reputational or legal risks. At Boston Consulting Group (BCG), we've been assisting numerous clients in navigating the LLM/GenAI landscape since late 2022. Unlike other AI applications, we've encountered challenges in finding suitable tools for testing LLM/GenAI systems. In response, we've developed an open-source Python package, ARTKIT, designed to facilitate automatic and scalable testing of LLMs/GenAI. In this talk, we'll share BCG's broader Responsible AI(https://www.bcg.com/capabilities/arti...) efforts and introduce ARTKIT(https://github.com/BCG-X-Official/artkit) to the PyData community. PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome! 00:10 Help us add time stamps or captions to this video! See the description for details. Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVi...