The Gap From Prototype to Production скачать в хорошем качестве

The Gap From Prototype to Production

Speaker: Ian Yu, Machine Learning Engineer, Groupby Inc Abstract: 2023 was a good year for prototyping LLM-based applications, but 2024 is a great year for productionizing them. However, going into production, there are many unforeseen questions and challenges. These include decisions between managed solutions and custom implementations, balancing the rigour of experimentation with the speed of application development, ensuring maintainability post-deployment, and evaluating and optimizing systems. Common issues also include LLM output inconsistencies at scale that were not apparent during prototyping, tightly coupled systems that are hard to pivot or modify, unclear evaluation objectives, and misaligned product-user fit. In this workshop, we will address these questions and challenges at the implementation level, including: Strategically align LLMs with product design and application logic Empirical tips on designing LLM chaining What incorporating LLMs with non-LLMs in a system looks like Application maintainability post-deployment Discuss various touch points for build vs. buy, such as prompt versioning, orchestration, vector databases Trends and predictions spurred by increased focus on production work This workshop also includes a little hands-on work to demonstrate bite-sized system