AI for AI Scotiabank's Award Winning ML Models скачать в хорошем качестве

AI for AI Scotiabank's Award Winning ML Models

Speakers: Narcisse Torshizi, Data Scientist/ Data Science Manager, Scotiabank Andres Villegas, Data Scientist Manager, Scotiabank Abstract: A brief overview of four innovative models that power and improve a chatbot solution Last year, Scotiabank was awarded the 2023 Digital Transformation Award by IT World Canada for our customer support chatbot. This achievement was made possible through the implementation of auxiliary AI models that helped the team develop the chatbot ("AI for AI"). These auxiliary models enabled the automation of the conversations review, supported NLU training, and allowed for scalability as the adoption of the chatbot increased. Besides, we have recently leveraged LLMs for summarizing chatbot interactions when a chatbot session is handed over to an agent (when the chatbot cannot fulfil the customer’s request). The chatbot solutions that we have developed and deployed is a result of combining various machine learning and statistical models. These models handle distinct aspects of natural language understanding, processing, and evaluation. Launching a new chatbot with no previous data puts immense pressure on the sustaining teams to detect, classify, and fix issues in the chatbot. In the absence of out of the box solutions the team came up with the concept of building auxiliary AI models to sustain the chatbot (AI for AI). We will describe the major features and achievements of four models that sustain our award-winning chatbot: Luigi, EVA, Peach and GenAI summarization. Luigi is a machine learning model that takes the confidence threshold of the chatbot's answers as either correct or incorrect. It uses a supervised learning approach to learn from the feedback of human reviewers and adjust the threshold accordingly. EVA is a machine learning classification model that processes customer inputs to predict their intent. It works in conjunction with Google Dialogflow. Peach is a natural language understanding model focused on similarity analysis. It supports AI trainers by evaluating whether training utterances positively influence the performance of the Dialogflow machine learning model. Finally, Our First GenAI feature helps summarization of the chat and capturing key details of each conversation, including account information and transaction specifics. This information is then sent to an agent, reducing the initial workload by an impressive 71%. On average, summaries are a mere 48 words, compared to the original 166-word conversations. By utilizing these models, the team tapped into a database of curated data, reducing manual labor by thousands of hours in maintaining the organization's chatbot. This enabled the chatbot to rapidly enhance its performance after launch, resulting in improved call containment, customer satisfaction, and ultimately, recognition with the 2023 Digital Transformation Award. These models handle different aspects of natural language processing and evaluation and work together to provide a seamless and satisfying customer experience.