Benchmarking 2000+ Cloud Servers for GBM Model Training and LLM Inference Speed скачать в хорошем качестве

Benchmarking 2000+ Cloud Servers for GBM Model Training and LLM Inference Speed

🔊 Recorded at PyData Berlin 2025, https://2025.pycon.de/program/ZLJRNN/ 🎓 Benchmarking 2000+ cloud servers across 5 vendors to find the most cost-efficient instances for LLM inference and ML workloads. Speakers: Gergely Daroczi Description: Spare Cores is an open-source, vendor-independent ecosystem that benchmarks cloud server performance across multiple providers. This talk presents results from testing over 2,000 server types across AWS, GCP, Azure, Hetzner, and other vendors to evaluate their suitability for serving Large Language Models from 135M to 70B parameters. The project measures inference speed for both prompt processing and text generation across varying token lengths, helping users identify the optimal instance type for their specific LLM serving needs. All data, tools, and benchmarks are openly published, including database dumps, APIs, and SDKs. The presentation covers the methodology, challenges in data collection, and surprising insights—such as how modern AMD CPUs can sometimes outperform GPU instances on a cost-efficiency basis for certain workloads. Future plans include expanding benchmarks to include GBM model training across popular frameworks like XGBoost and LightGBM. ⭐️ About PyData Berlin: Since 2014, PyData Berlin has brought together the international Python and data community. With monthly meetups and a yearly conference, it connects data engineers, scientists, and tool-builders to share ideas, explore open-source technologies, and drive innovation together. Follow us: • LinkedIn:   / pydata-berlin   • X: https://www.x.com/pydataber Links: • Conference website: https://berlin.pydata.org/ • Other sessions: https://berlin.pydata.org/conferences... The conference is organized by • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives.