Tülu 3 from AI2: Full open-source fine-tuning recipe for LLMs скачать в хорошем качестве

Tülu 3 from AI2: Full open-source fine-tuning recipe for LLMs

Tülu 3 is a fully open-source LLM built by AI2 (the Allen Institute of AI) by post-training Llama models. Their fine-tuning recipe includes SFT (Supervised Fine-Tuning) for instruction fine-tuning, DPO (Direct Preference Optimization) for preference fine-tuning and RLVR (Reinforcement Learning with Verifiable Rewards). Paper: https://arxiv.org/abs/2411.15124 GitHub: https://github.com/allenai/open-instruct Models: https://huggingface.co/collections/al... Datasets: https://huggingface.co/collections/al... Datasets spreadsheet: https://docs.google.com/spreadsheets/... Watch more LLM deep dives here:    • 8 Timeless tips for training LLMs | Become...   Blogpost version of this video on Substack: https://juliaturc.substack.com/p/tulu... 00:00 Intro 00:53 LLM training nowadays 02:47 Instruction fine-tuning (SFT) 05:51 Preference fine-tuning (DPO) 09:10 Reasoning fine-tuning (RLVR) 10:39 Synthetic data generation 12:29 Evaluation & results #Tulu3 #LLMFineTuning #OpenSourceAI #DeepLearning #MachineLearning #AIResearch #AllenAI #LLaMA #DPO #PPO #AI