Fine-tuning LLMs with PEFT and LoRA - Gemma model & HuggingFace dataset скачать в хорошем качестве

Fine-tuning LLMs with PEFT and LoRA - Gemma model & HuggingFace dataset

In this video, I give a code walkthrough of finetuning the latest Gemma 2B parameter model from Google in the HuggingFace ecosystem. I have created a new Databricks-dolly-mini dataset derived from the Databricks-dolly-15k dataset. The newly created dataset is then used to fine-tune Gemma 2b to visualize the training progress and the results. ⌚️ ⌚️ ⌚️ TIMESTAMPS ⌚️ ⌚️ ⌚️ 0:00 - Intro 1:24 - Preliminaries & Installation 2:44 - Run inference on pre-trained Gemma-2b 4:44 - Motivation for Parameter Efficient Fine-tuning (PEFT) 7:58 - Create a custom dataset for finetuning 11:58 - Tips for fine-tuning (parameters) 19:29 - Supervised Fine-tuning 20:30 - Training visualization & Interpretations 23:04 - Conclusion RELATED LINKS Colab Notebook: https://github.com/ai-bites/generativ... Google's Gemma: https://blog.google/technology/develo... Databricks Dolly 15k dataset: https://huggingface.co/datasets/datab... New Databricks Dolly mini dataset: https://huggingface.co/datasets/ai-bi... LoRA paper: https://arxiv.org/abs/2106.09685 My LoRA video:    • LoRA (Low-rank Adaption of AI Large Langua...   My QLoRA video:    • QLoRA paper explained (Efficient Finetunin...   MY KEY LINKS YouTube:    / @aibites   Twitter:   / ai_bites​   Patreon:   / ai_bites​   Github: https://github.com/ai-bites​ WHO AM I? I am a Machine Learning researcher/practitioner who has seen the grind of academia and start-ups equally. I started my career as a software engineer 16 years ago. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the now happening AI revolution started. Life has changed for the better ever since. #machinelearning #deeplearning #aibites