Taking Control of LLM Outputs: An Introductory Journey into Logits скачать в хорошем качестве

Taking Control of LLM Outputs: An Introductory Journey into Logits

🔊 Recorded at PyCon DE & PyData 2025, April 23, 2025 https://2025.pycon.de/program/VDG9YG/ 🎓 A deep dive into controlling LLM outputs through logit manipulation, exploring sampling methods and structured generation techniques for production-ready implementations. Speakers: Emek Gözlüklü Description: In this technical presentation, Emek Gözlüklü explores the mechanisms for controlling Large Language Model (LLM) outputs through logit manipulation and structured generation approaches. The talk begins with foundational concepts of token processing and embedding matrices, explaining how models generate text through logit values - the raw numerical scores representing token probabilities. Gözlüklü demonstrates various sampling methods including greedy sampling, multinomial sampling with temperature control, top-K, top-P (nucleus sampling), and beam search, illustrating their effects on output generation. The presentation emphasizes the practical importance of structured generation for production systems, highlighting how constraint decoding and token masking can ensure outputs follow specific patterns or formats. Two key libraries, Outlines and Guidance, are examined for implementing structured generation with local LLMs. The talk concludes with real-world applications, including uncertainty quantification and evaluation metrics based on logit values. Throughout the presentation, examples utilize the LLAMA-3B-instruct model to demonstrate concepts, though the principles apply across different model architectures. The primary takeaway is the practical value of understanding and controlling LLM outputs for building reliable AI systems. ⭐️ About PyCon DE & PyData: The PyCon DE & PyData conference unite the Python, AI, and data science communities, offering a unique platform for collaboration and innovation. The PyCon DE & PyData 2025 conference, provided an exceptional experience, fostering deeper connections within the Python community while showcasing advancements in AI and data science. Attendees enjoyed a diverse and engaging program, solidifying the event as a highlight for Python and AI enthusiasts nationwide. Follow us: • LinkedIn:   / 28908640   • X: https://www.x.com/pyconde Links: • Conference website: http://pycon.de • Other sessions: https://2025.pycon.de/talks/ The conference is organized by • Python Softwareverband e.V.: http://pysv.org • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: Python Softwareverband e.V.: PySV is a non-profit that promotes the use and development of Python in Germany through events, education, and advocacy, fostering an open Python community. NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives. www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.