Self-Supervised Learning Review: From SimCLR to DINOv2 скачать в хорошем качестве

Self-Supervised Learning Review: From SimCLR to DINOv2

In this video, I explored the architectures of SimCLR, BYOL, SwAV, and iBOT, and how they directly contributed to the development of DINOv2. I also touched on related topics, such as contrastive loss, Siamese networks, and avoiding collapse. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 🔗 notes: https://mashaan14.github.io/YouTube-c... ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬    • Inside DINOv2: Architecture Analysis + CIF...   ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ website https://mashaan14.github.io/mashaan/ github https://github.com/mashaan14 X   / mashaan_14   linkedin   / mashaan   ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ sponsor this channel on GitHub Sponsors: https://github.com/sponsors/mashaan14 ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Chapters: 00:00 start 00:12 what is SSL? 02:22 SimCLR 05:50 visualizing SimCLR 09:03 why a projection head? 11:54 what's wrong with negative pairs? 14:14 BYOL 17:00 my experience with Siamese networks 20:17 SwAV 25:43 DINOv1 30:22 avoiding collapse in DINOv1 31:56 iBOT 36:51 DINOv2 ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ References: Chen, Ting, et al. "A simple framework for contrastive learning of visual representations." International conference on machine learning. PmLR, 2020. Grill, Jean-Bastien, et al. "Bootstrap your own latent-a new approach to self-supervised learning." Advances in neural information processing systems 33 (2020): 21271-21284. Caron, Mathilde, et al. "Unsupervised learning of visual features by contrasting cluster assignments." Advances in neural information processing systems 33 (2020): 9912-9924. Caron, Mathilde, et al. "Emerging properties in self-supervised vision transformers." Proceedings of the IEEE/CVF international conference on computer vision. 2021. Zhou, Jinghao, et al. "ibot: Image bert pre-training with online tokenizer." arXiv preprint arXiv:2111.07832 (2021). Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023). ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #DINOv2 #DINO #iBOT #SimCLR #BYOL #SwAV #IJEPA #VJEPA #SelfSupervised #TransferLearning #LVD142M #ViT #VisionTransformer #PyTorch #DeepLearningTutorial #MachineLearningProject #AIResearch #CodingTutorial