Implementing GPT-2 From Scratch (Transformer Walkthrough Part 2/2) скачать в хорошем качестве

Implementing GPT-2 From Scratch (Transformer Walkthrough Part 2/2)

See part 1 here: What is a transformer? https://neelnanda.io/transformer-tuto... Template notebook: https://neelnanda.io/transformer-temp... Solution notebook: https://neelnanda.io/transformer-solu... If you enjoyed this, I expect you'd enjoy learning more about what's actually going on inside these models and how to reverse engineer them! Check out: A Comprehensive Mechanistic Interpretability Explainer & Glossary: https://www.neelnanda.io/glossary Concrete Steps for Getting Started in Mechanistic Interpretability: https://www.neelnanda.io/getting-started 200 Concrete Open Problems in Mechanistic Interpretability: https://www.neelnanda.io/concrete-ope... Further resources: The transformers section of my MI explainer: https://dynalist.io/d/n2ZWtnoYHrU1s4v... My TransformerLens library for doing mechanistic interpretability research on GPT-2 style language models: https://github.com/neelnanda-io/Trans... My walkthrough of A Mathematical Framework for Transformer Circuits, for a deeper dive into how to think about transformers:    • A Walkthrough of A Mathematical Framework ...   Check out these other intros to transformers for another perspective: Jay Alammar's illustrated transformer: https://jalammar.github.io/illustrate... Timestamps: 00:00 Intro 04:01 Recap 05:03 Setup 06:04 LayerNorm 23:35 Embedding 30:07 Attention 51:22 MLP 54:00 Transformer Block 56:40 Unembedding 58:50 Full Transformer 1:01:47 Trying it out 1:11:05 Training