DINO in PyTorch скачать в хорошем качестве

DINO in PyTorch

In this video, we implement the DINO scheme in PyTorch. It is a selfsupervised learning framework for images proposed in the paper "Emerging Properties in Self-Supervised Vision Transformers". The goal is to train a network that is able to represent images in a meaningful way without using human labels. Among other things, if applied to a Vision Transformer network one can inspect the attention weights and get surprisingly good results. Last but not least, this video also contains short tutorials on weight normalization and PyTorch buffers. Paper: https://arxiv.org/abs/2104.14294 Official code: https://github.com/facebookresearch/dino Imagenette dataset: https://github.com/fastai/imagenette Vision Transformer code: https://github.com/rwightman/pytorch-... Code from the video: https://github.com/jankrepl/mildlyove... 00:00 Intro + disclaimer 00:50 High-level explanation 02:38 Imagenette - our dataset 03:11 timm - our Vision Transformer 03:29 Augmentations implemented 08:01 Augmentations visualized 08:48 Head implementation 11:12 [IPython] Weight normalization 15:48 Head implementation continued 16:54 MultiCropWrapper implementation 18:48 [IPython] Buffers in PyTorch 21:13 Loss implementation 25:50 Gradient clipping implementation 26:24 KNN classifier implementation 28:39 TensorBoard embedding preparation 30:01 Training script 37:35 Launch training 38:04 Results: Supervised model 39:17 Results: Selfsupervised model 41:17 Results: Visualizing attentions 43:39 Outro If you have any video suggestions or you just wanna chat feel free to join the discord server:   / discord   Twitter:   / moverfitted   Credits logo animation Title: Conjungation · Author: Uncle Milk · Source:   / unclemilk   · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600