How to Create a Custom Serving Runtime in KServe ModelMesh to S... Rafael Vasquez & Christian Kadner скачать в хорошем качестве

How to Create a Custom Serving Runtime in KServe ModelMesh to S... Rafael Vasquez & Christian Kadner

How to Create a Custom Serving Runtime in KServe ModelMesh to Serve Your Models - Rafael Vasquez & Christian Kadner, IBM KServe ModelMesh comes with a few built-in serving runtimes to serve models in common ML frameworks. However, at times there is a need to serve models in custom runtimes, specially built to satisfy various requirements. In this talk, we will introduce ModelMesh key features, and then share hands-on experience of building a new container image with support for the desired framework. The following will be covered: - Overview of ModelMesh - Demo of loading models in existing model server - Metrics and dashboard - How to create a new custom serving runtime - Demo of running prediction using custom serving runtime