LLM (Parameter Efficient) Fine Tuning - Explained! скачать в хорошем качестве

LLM (Parameter Efficient) Fine Tuning - Explained!

Parameter efficient fine tuning is increasingly important in NLP and genAI. Let's talk about it. RESOURCES [1 📚] RNNs were the SOTA for sequence tasks: https://arxiv.org/pdf/1409.0473 [2 📚] Then transformers came on the scene: https://arxiv.org/pdf/1706.03762 [3 📚] Pretraining and Finetuning architectures like BERT came along: https://arxiv.org/pdf/1810.04805 [4 📚] But LLMs are huge: https://informationisbeautiful.net/vi... [5 📚] Few shot learning by GPT-3 tries to address the issue: https://arxiv.org/pdf/2005.14165 [6 📚] Parameter Efficient Transfer Learning reduces the trainable parameters via additive adapters (the first PEFT technique): https://arxiv.org/pdf/1902.00751 [7 📚] Since 2019, there have been many PEFT techniques introduced: https://arxiv.org/pdf/2312.12148 [8 📚] Other notable techniques include prefix-tuning: https://arxiv.org/pdf/2101.00190 [9 📚] And LoRA: https://arxiv.org/pdf/2106.09685 [10 📚] And a quantized version of LoRA called QLoRA: https://arxiv.org/pdf/2305.14314 [11 📚] We see these adapters in use in LLMs today like Llama: https://arxiv.org/pdf/2303.16199 ABOUT ME ⭕ Subscribe: https://www.youtube.com/c/CodeEmporiu... 📚 Medium Blog:   / dataemporium   💻 Github: https://github.com/ajhalthor 👔 LinkedIn:   / ajay-halthor-477974bb   PLAYLISTS FROM MY CHANNEL ⭕ Deep Learning 101:    • Deep Learning 101   ⭕ Natural Language Processing 101:    • Natural Language Processing 101   ⭕ Reinforcement Learning 101:    • Reinforcement Learning 101   Natural Language Processing 101:    • Natural Language Processing 101   ⭕ Transformers from Scratch:    • Natural Language Processing 101   ⭕ ChatGPT Playlist:    • ChatGPT   CHAPTERS 0:00 Introduction 1:00 Pass 1: What & Why PEFT 6:27 Quiz 1 7:26 Pass 2: Details 16:20 Quiz 2 17:11 Pass 3: Performance Evaluation 20:49 Quiz 3 21:43 Summary MATH COURSES (7 day free trial) 📕 Mathematics for Machine Learning: https://imp.i384100.net/MathML 📕 Calculus: https://imp.i384100.net/Calculus 📕 Statistics for Data Science: https://imp.i384100.net/AdvancedStati... 📕 Bayesian Statistics: https://imp.i384100.net/BayesianStati... 📕 Linear Algebra: https://imp.i384100.net/LinearAlgebra 📕 Probability: https://imp.i384100.net/Probability OTHER RELATED COURSES (7 day free trial) 📕 ⭐ Deep Learning Specialization: https://imp.i384100.net/Deep-Learning 📕 Python for Everybody: https://imp.i384100.net/python 📕 MLOps Course: https://imp.i384100.net/MLOps 📕 Natural Language Processing (NLP): https://imp.i384100.net/NLP 📕 Machine Learning in Production: https://imp.i384100.net/MLProduction 📕 Data Science Specialization: https://imp.i384100.net/DataScience 📕 Tensorflow: https://imp.i384100.net/Tensorflow