Low latency Neural Network Inference for ML Ranking Applications Yelp Case Study скачать в хорошем качестве

Low latency Neural Network Inference for ML Ranking Applications Yelp Case Study

Speakers: Ryan Irwin, Engineering Manager, Yelp Inc. Ryan Irwin is a senior engineering manager at Yelp. He leads the teams responsible for the ML Platform, which covers ML computing, feature engineering, model training, and model inference. Ryan has a Ph.D. in Computer Engineering from Virginia Tech. Rajvinder Singh, Sr Product Manager, Yelp Inc. Rajvinder is currently leading product for the CoreML Group, and was previously an Engineering Manager at Etsy where he lead the ML Platform team. Abstract: At Yelp, we train and deploy models for a variety of business applications requiring low-latency model inference. At first we focused on streamlining support for XGboost and LR models built in Spark to support business recommendations, search, ads, restaurants, and trust & safety use-cases. However, we didn’t have a way of supporting low-latency neural network models with Tensorflow. Such models usually relied on batched model inference in support of models used for photo classification [1] and popular dishes [2]. In this talk, we give an architectural overview of our ML Platform and how we overhauled it to support neural network models in low-latency ranking applications. We cover how we built in the capabilities to train and deploy Tensorflow-based models using MLEAP and cataloged them in MLFlow. We also discuss the deployment plugin that was using Elasticsearch and how this transitioned to using Yelp’s own near-real time search (Nrtsearch) [3] open-source framework. Lastly, we cover the issues faced along the way in terms of latency and model performance, including how we incorporate embedded features in the model. [1] https://engineeringblog.yelp.com/2015... [2] https://engineeringblog.yelp.com/2019... [3] https://engineeringblog.yelp.com/2021...