A SRE’s Guide to LLMOps: Deploying and Managing AI⧸ML Workloads using Kubernetes - Harsh Mishra скачать в хорошем качестве

A SRE’s Guide to LLMOps: Deploying and Managing AI⧸ML Workloads using Kubernetes - Harsh Mishra

This talk was originally part of a webinar hosted by ‪@MirantisUS‬ For Site Reliability Engineers (SREs) and DevOps professionals diving into the world of Large Language Models (LLMs), Retrieval-Augmented Generation (RAGs), and GenAI, the lack of standardization, resources, and knowledge can quickly become overwhelming. DevOps and SRE practices have evolved beyond just infrastructure and delivery speed; they now also involve the complexities of GenAI, especially LLMs. While it might seem easy for an inexperienced SRE to treat this as a typical MLOps issue, it soon becomes evident that LLMOps presents a whole new set of challenges, right from the start. In this webinar, Harsh Mishra, an SRE at One2N Consulting, discusses the unique challenges and solutions of LLMOps from an SRE viewpoint. The best practices he outlines will provide viewers with a great jumping off point for LLMOps, as well as future-proofing their cloud strategy to include the rapidly evolving field of GenAI and LLMs. CHAPTERS 0:00 - Intro 0:24 - Why SREs need to worry about LLMs 1:11 - One2N's journey towards true LLMOps 15:30 - One2N's successes & challenges with K8s LLMOps 18:35 - MLOps vs LLMOps: Infra, Deployment & Architecture 19:31 - VMs to Kubernetes for LLMOps 23:07 - Vector Storage: index vs database 25:12 - Deep dive into One2N's Lab setup (real-world use case) 28:58 - Key takeaways to get started with LLMOps 31:23 - Outro