V-JEPA for Visual Representation Learning from Video скачать в хорошем качестве

V-JEPA for Visual Representation Learning from Video

Joint-Embedding Predictive Architectures (JEPA) are trained to predict the representations of an input 𝑦 from the representation of another input 𝑥 (e.g., copy of 𝑦 with some regions masked). The additional variable 𝑧 provides the predictor with information about the transformation (or corruption) that computes 𝑦 from 𝑥. Without using pretrained encoders, Video-JEPA (V-JEPA) learns versatile visual representation from video through feature prediction in a self-supervised manner, and the masked inputs 𝑥 and 𝑦 in a video clip are complement. This video introduces the architecture of V-JEPA model and how to design a L1 loss to train the V-JEPA model.