Deploying AI Runtimes скачать в хорошем качестве

Deploying AI Runtimes

Eric explains how to deploy the Ecosystem AI runtime into production with a focus on Kubernetes/OpenShift, covering three deployment patterns: permanently running runtime pods that accept pushed configuration updates (optionally compiling custom Java pre/post-scoring logic or reward functions), a recommended setup adding a separate Runtime MCP container to expose MCP protocol tools for agents/LLMs, enable MLflow model integration, and support custom Python APIs, and a less agile approach using versioned images with embedded configuration for strict rollback needs. Demos Python notebook automation for dynamic and static model deployments, including authentication, syncing and compiling code, optional Cassandra config and API config updates, pushing configuration, and testing calls, and discusses scaling considerations and using managed LLM services like Amazon Bedrock rather than running LLMs inside Kubernetes. 00:54 Deployment Patterns Overview 02:54 Always-On Runtime Pods 04:57 Custom Java Logic Builds 07:03 Runtime MCP Benefits 10:19 Versioned Image Deployments 13:02 Demo Setup Notebooks 13:15 Dynamic Model Push Workflow 19:17 Static Models with MLflow 21:52 Scaling and LLM Integration Q&A