Lets Reproduce the Vision Transformer on ImageNet скачать в хорошем качестве

Lets Reproduce the Vision Transformer on ImageNet

Code: https://github.com/priyammaz/PyTorch-... Today we will be doing a full reproduction script of the Vision Transformer on ImageNet! I hope you already know Attention, if you don't please check out this video here    • Attention is All You Need: Ditching R...   The trick to training a ViT from scratch has less to do with the model and more to do with Data Augmentation. After we build the model we will build a distributed training pipeline with MixCut, CutMix and RandAug. The model trained for 300 epochs reached roughly 80% Top1 Accuracy which is close to the results reported by PyTorch! Timestamps: 00:00:00 Introduction 00:06:15 Images to Patch Embeddings 00:17:20 Local vs Global Image Comprehension 00:21:00 Self-Attention 00:36:00 MultiLayer Perceptron 00:41:10 Encoder Block 00:45:20 CLS Token vs Pooling 00:50:10 Vision Transformer Implementation 01:09:40 The Importance of Augmentation! 01:12:00 Implement Training Augmentation Pipeline 01:24:15 Mixup and CutMix 01:35:28 Testing Augmentation 01:40:25 Calculate TopK Accuracy 01:49:48 Distributed Training Script 2:08:15 Testing the Training Script 2:10:28 Results Socials! X   / data_adventurer   Instagram   / nixielights   Linkedin   / priyammaz   🚀 Github: https://github.com/priyammaz 🌐 Website: https://www.priyammazumdar.com/