One API to Rule Them All? LiteLLM in Production скачать в хорошем качестве

One API to Rule Them All? LiteLLM in Production

🔊 Recorded at PyData Berlin 2025, https://2025.pycon.de/program/NUNXEV/ 🎓 Real-world lessons from using LiteLLM in production: what worked, what didn't, and when you should consider alternatives. Speakers: Alina Dallmann Description: Alina Dallmann shares practical insights from implementing LiteLLM in a production RAG system for a university chatbot. LiteLLM promises a unified interface across multiple LLM providers, but how does it perform in real-world scenarios? This talk covers the implementation of LiteLLM's proxy server for a privacy-preserving chatbot serving students and employees. Alina walks through key features including automatic cost tracking, budget management, and load balancing across multiple models. She demonstrates how LiteLLM simplifies provider switching and handles token budgets through PostgreSQL integration. However, the talk also addresses significant challenges: limitations in custom budgeting requirements, delays in new model support, occasional bugs in new features, and the trade-offs of abstraction layers. Alina provides honest assessments of when LiteLLM excels—multi-provider setups with governance needs—and when alternatives like OpenRouter or TensorZero might be better choices. Essential viewing for developers considering LiteLLM beyond simple prototypes. ⭐️ About PyData Berlin: Since 2014, PyData Berlin has brought together the international Python and data community. With monthly meetups and a yearly conference, it connects data engineers, scientists, and tool-builders to share ideas, explore open-source technologies, and drive innovation together. Follow us: • LinkedIn:   / pydata-berlin   • X: https://www.x.com/pydataber Links: • Conference website: https://berlin.pydata.org/ • Other sessions: https://berlin.pydata.org/conferences... The conference is organized by • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives.