AI Document Ingestion and Querying with KAITO RAG Engine on Azure Kubernetes скачать в хорошем качестве

AI Document Ingestion and Querying with KAITO RAG Engine on Azure Kubernetes

I demonstrate the capabilities of the Kubernetes AI Toolchain Operator running on Azure Kubernetes Service (AKS). In particular, I showcase the KAITO RAG Engine, which exposes an API for ingesting document text into a vector store and enabling question-answering over the indexed content. The demo application is built with Streamlit and provides the ability to: ingest documents create and manage a prompt library send prompts to the RAG engine to query the vector index This demo uses the open-source Phi‑4 large language model to generate responses grounded in the retrieved documents. The Phi-4 model is also deployed and hosted on AKS using the Kubernetes AI Toolchain Operator. I compare different prompting scenarios using the Phi-4 LLM with and without the RAG engine to illustrate the effectiveness of the Retrieval‑Augmented Generation (RAG) approach. The Kubernetes AI Toolchain Operator is an add-on for AKS that simplifies deploying open-source LLMs by automatically provisioning GPU nodes and inference endpoints. https://github.com/kaito-project/kaito Visit my github repo to see my infra setup and applicaiton code: https://github.com/RoyKimYYZ/aks-demo... Intro 0:00 Demo Architecture and Data Flow 2:05 Streamlit App Demo 5:50 Wrap up 17:01 My blog: www.roykim.ca